Add Prometheus monitoring to Pages

It would be helpful to add Prometheus metrics to GitLab Pages to track how long it takes:

1. Update a list,
2. Number of served domains,
3. Last update time,
4. Number of processed requests,
5. Number of currently being processed requests.

mentioned in issue gitlab-com/infrastructure#1058 (closed)

assigned to @zj

@ayufan @nick.thomas If I understand correctly, these metrics will be exported through HTTP. Given we can listen to multiple HTTP and HTTPS ports/adresses were would it be best to expose this?

@zj I would add a new -listen-prometheus option which would be handled in the same way as the existing ones.

Things are a little complicated as the process opens the ports (as they may be on port 80 or so, requiring privileges), then optionally drops privileges and chroot()s and executes itself again, passing the file descriptors along. I got to work all this out while being interviewed for the job 🕐 !

I don't think we want to put the prometheus endpoint on the existing listeners - that way lies madness, and it's not what we did in gitlab-workhorse either. That discussion started here: https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/86#note_18935238

@nick.thomas This is not a problem if you want to listen on unprivileged port. This was done for privileged ones. So we can go easy route for metrics.

Last update time

I don't think this is needed, and Ill skip it in the first implementation. Given we have no clue when and how often scraping will happen, and we're going to store the number of updates anyway, this should be fine.

@zj

Can we just store UTC time when update did happen and allow this to be scraped?

mentioned in merge request !11 (merged)

I'd really prefer consistency when handling listen()s, the framework for it is already all there after all.

mentioned in commit 1e31fac9

closed via merge request !11 (merged)

closed via commit 1e31fac9