Pass a list of custom domains to the gitlab-pages daemon through the .update file

Nick Thomas requested to merge (removed):pass-domain-map-to-pages into master

What does this MR do?

Modify the pages update process so that useful information is written to the .update file, instead of just a random string

Are there points in the code the reviewer needs to double check?

I still need to reason through the concurrency aspects of this. Will we need an exclusive lock on updating this file? Forced in-order execution of jobs?

The performance impact of this is expected to be fairly limited. The file we write increases linearly in size as more custom domains are added. has around 20,000 domains at the moment, which equates to a 600KiB file:

Why was this MR needed?

Currently, the pages daemon needs to read the config.json file for a project before it can serve custom domains. For large installations, this amounts to many thousands of files, and it can take several minutes to read all of them. On startup, this causes 404 responses for domains until the background worker has completed at least one full run

A domain -> project map in a single file allows us to load configuration at first request instead, improving the user experience

Screenshots (if relevant)

Does this MR meet the acceptance criteria?

What are the relevant issue numbers?

Related to

Merge request reports