Add metrics for queueing mechanism
requested to merge tmaczukin/gitlab-workhorse:feature/add-metrics-for-queueing-mechanism into master
This MR adds metrics that will be helpful to track what is happening with queueing (currently used for POST /ci/api/v1/builds/register.json
).
Example of added metrics:
gitlab_workhorse_queueing_busy{queue_name="ci-api-job-requests"} 1
gitlab_workhorse_queueing_errors{queue_name="ci-api-job-requests",type="queueing_timedout"} 10
gitlab_workhorse_queueing_errors{queue_name="ci-api-job-requests",type="too_many_requests"} 15
gitlab_workhorse_queueing_limit{queue_name="ci-api-job-requests"} 1
gitlab_workhorse_queueing_queue_limit{queue_name="ci-api-job-requests"} 100
gitlab_workhorse_queueing_waiting{queue_name="ci-api-job-requests"} 81
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="0.01"} 0
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="0.02"} 0
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="0.05"} 1
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="0.1"} 3
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="0.2"} 10
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="0.5"} 67
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="1"} 444
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="2"} 1375
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="5"} 1559
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="10"} 1559
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="30"} 1569
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="60"} 1569
gitlab_workhorse_queueing_waiting_time_bucket{queue_name="ci-api-job-requests",le="+Inf"} 1569
gitlab_workhorse_queueing_waiting_time_sum{queue_name="ci-api-job-requests"} 2115.3471269120005
gitlab_workhorse_queueing_waiting_time_count{queue_name="ci-api-job-requests"} 1569