Skip to content
Snippets Groups Projects
New issue look: Off

Improve ElasticSearch metrics and monitoring

    • Closed Issue created by Nick Thomas @nick.thomas

    In discussion with @pcarranza, he raised the issue of our ElasticSearch operations being fairly opaque. We don't know how ES is performing, and so are reduced to making educated guesses about why it causes us problems on GitLab.com

    So, we need to add metrics and instrumentation, and bring it all together in a neat Grafana dashboard. Gathering these numbers and putting them somewhere visible allows us to evaluate the impact of changes, and work out what changes to make in the first place.

    It's possible some of the metrics we're interested in are already being stored, and just need exposing. We need to know what we want to see!

    ES operations can be split into two categories: indexing, and searching. @maratkalibek do you have any thoughts on what we want to see here?

    Indexing, I think we want to know the rate of indexing operations, and have statistics (p50, p90, p99) on how long those operations take, binned by the type of thing we're indexing (wiki, commit, issues, etc).

    Search, we definitely want to know the search rate (I expect we can find this already) and the p50/90/99 on how long elasticsearch queries take to return. Perhaps we have this already as well.

    Designs

      Child items ...

      2. Activity

      Please register or sign in to reply