Using metrics shipped in omnibus
@sytses asked this question today on the infra call: "Can we start also using the metrics that are collected by GitLab Omnibus by default?", and the answer was effectively no, no-one has spent time on this.
So what metrics are collected by GitLab Omnibus by default? I was able to find the following:
- using Prometheus for monitoring: https://docs.gitlab.com/ce/user/project/integrations/prometheus.html#getting-started-with-prometheus-outside-of-kubernetes ==> we have been using this since before it was shipped with GitLab.
- we ship Prometheus exporters for Kubernetes, NGINX, HAProxy, and Amazon Cloud Watch, https://docs.gitlab.com/ce/user/project/integrations/prometheus_library/metrics.html
- For NGINX we have the following default metrics (https://docs.gitlab.com/ce/user/project/integrations/prometheus_library/nginx.html) :
- throughput
- latency
- http error rate
- For HAProxy we have the following default metrics (https://docs.gitlab.com/ce/user/project/integrations/prometheus_library/haproxy.html):
- throughput
- http error rate
- It looks like additional information can be gained if we are running on kubernetes, see e.g. https://docs.gitlab.com/ce/ci/environments.html#monitoring-environments but since we are not on kubernetes yet that does not apply.
So what is shipped that we are not already measuring?
- In the Fleet Overview dashboard, https://performance.gitlab.net/dashboard/db/fleet-overview?refresh=5m&orgId=1, there is a dropdown for HAProxy; looks like we record HTTP code rates and Sidekiq running jobs. BUT, I don't see a similar dropdown for NGINX. Would love to see NGINX latency...
And what do we measure that is not shipped?
- Kind of a lot... so perhaps the better question is once we ship a metric, how do we make sure we are seeing it through our use of the Omnibus provided GitLab install instead of through our existing monitoring infrastructure?
Edited by Ernst van Nierop