Package and deploy InfluxDB-Relay, InfluxDB Exporter

Description

We have a major gap today in the Prometheus based metrics, which is a gap in data from our Ruby code. We have a long term to build an exporter for Unicorn, but this is going to take time to build and then instrument our code for support. In the interim, it would be good to be able to fill this gap with data that is output to InfluxDB.

Proposal

We already have InfluxDB based metrics being output from our Ruby code, and can use that as a source of metrics while work on the Unicorn Exporter continues. This will allow us to meet and exceed what is available today via InfluxDB, so that we can deprecate that monitoring method and fully move onto Prometheus.

Note: For customers who want to continue to use InfluxDB, there is support within InfluxDB to read data from Prometheus.

To achieve this, we can leverage InfluxDB-Relay to provide an aggregation and multi-output relay for InfluxDB metrics streams. It is able to split an InfluxDB UDP stream to both InfluxDB Exporter and, if configured, InfluxDB.

We can then have configured an InfluxDB Exporter which receives one of the streams at all times, to provide an interface for Prometheus to scrape. The other stream from the relay can be directed at the existing customer InfluxDB server, if one was configured.

The challenge I think we will have, and need to quickly determine, is how we can smoothly and without interruption reconfigure the output directly from GitLab to the relay, and then move the current (and any future) configuration settings from GitLab to the relay.