Support specific colors for a given time series

In the Prometheus monitoring world, it is pretty common for a single query to return multiple time series.

The best example for this would be trying to illustrate the HTTP Request Rate in a single chart. We would want to have separate time series on the chart for 2xx, 4xx, and 5xx return types. We would 2xx to be green, 4xx to be perhaps yellow, and 5xx to be red. For Prometheus exporters, these are collected in a single metric, and with each response code specified as part of a label. (This behavior is very much the standard across most Prometheus metrics too.)

This same model is again used throughout Prometheus, for example it also used to fan out over individual CPU's in our existing k8s CPU metric. Currently we simply chose to summarize across all CPUs, but we could also break this data out. While this use case is not a strong from a customer value POV as the HTTP Responses, it does provide a much easier sample case as it is already part of the GDK Prometheus setup and requires no additional components.

Here is a sample query:

We modify our original k8s CPU query to summarize by the individual CPU, which now causes 4 time series to be returned each with a label specifying the CPU. Now we would want to replicate this type of behavior within GitLab as well, again for use cases like Request Rate by HTTP response codes.

We can of course specify the query today in the YML file, but we don't have the ability to specify color. Perhaps we can add this with something like the following format:

  - title: "CPU usage"
    required_metrics:
     - container_cpu_usage_seconds_total
    weight: 1
    queries:
    - query_range: 'avg(rate(container_cpu_usage_seconds_total{%{environment_filter}}[2m])) by (cpu) * 100'
      unit: Average (%)
      series:
        label: cpu
        - value: cpu00
          color: red
        - value: cpu01
          color: blue

• A new series: label to specify additional details on each series that the query would return.
• The series object would then contain a field for the label you would want to fan out over, and then key/value map for the each label value and corresponding color.
• I think we can hopefully use standard color names like "red" and "green" to keep things easy.
• Ideally this additional map should be optional. In the event there are values returned in the time series that aren't here, they should be assigned a color from a palette.
• The legend can then use the label value as the name of the series, and then show the corresponding value.

Design Proposal

Use the 500 numbers from our current color system. This would look like:

These are defined as variables already: $purple-500 $blue-500 $green-500 $orange-500 $red-500