Add alerts for git storage circuit breaker (!328) · Merge requests · GitLab.com / runbooks · GitLab

Do not update/delete: Banner broadcast message test data

Do not update/delete: Notification broadcast message test data

Bob Van Landuyt :neckbeard: requested to merge bvl-add-alerts-for-circuitbreaker into master Aug 18, 2017

This adds alerts for the state of the circuit breaker based on: https://prometheus.gitlab.com/graph?g0.range_input=1h&g0.expr=filesystem_circuitbreaker%7Benvironment%3D%22prd%22%7D&g0.tab=0

And alerts for the latency of access to the storage based on: https://prometheus.gitlab.com/graph?g0.range_input=1h&g0.expr=filesystem_circuitbreaker_latency_seconds%7Benvironment%3D%27prd%27%7D&g0.tab=0

Currently these values don't mean anything since the CB-feature is not enabled.

I'm not quite sure how to test these alerts.

Related to: https://gitlab.com/gitlab-com/infrastructure/issues/2409