[META] Run Registry separately from the rest of the fleet

Part of https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2441

TODO:

• Create the staging-base-lb-be, staging-base-be and staging-base-be-registry roles.
• Wire up api*.stg.gitlab.com to the internal lb and create a DNS record to point to the VIP.
• Install the registry certificate on the load balancer to terminate the TLS session.
• Create a sensible configuration to put in the gitlab-staging-be-registry role.
• Roll it out to staging
• Staging: Test!
  • Create a project, and push a image to the project
  • Pull an image from GitLab
  • Monitor the logs for errors/warnings
  • Consider inviting CI/CD team to do their own testing, with the CI. Consider inviting Support team too because they can stress test as well
• If tests look good, roll out to production!

Steps to containerise the service:

• Staging: If all is ok in production, try using the official Registry container and configure it on a node. Doesn’t have to be cheffed -> Do this together with Build
• Staging: Do another round of testing while using the containers.
• Staging: Setup working K8S cluster for Registry
• Staging: Test whether it works
• Production: Roll out

/cc @pcarranza @omame @ernstvn @joshlambert

mentioned in issue gitlab-org/omnibus-gitlab#2441 (closed)

changed the description

assigned to @omame

changed title from Run Registry separately from the rest of the fleet to [META] Run Registry separately from the rest of the fleet

I'm setting this as a meta because there is a looot to do here.

mentioned in issue #2084 (closed)

The backend roles have been created.

Next step here is to create the first back-end load balancer in staging.

added goal label

mentioned in issue #2149 (closed)

The backend load balancer and the registry node have been created on Monday. I'll track the progress on the configuration of the registry node (which isn't yet operational) in https://gitlab.com/gitlab-com/infrastructure/issues/2149.

We had some success with #2149 (closed) so the next steps are:

• Registry is running independently from the rest of the code base. Daniele to finish the configuration on the Rails side which is nearly done (semi-blocked by broken staging).
• Once that is in, get the staging fully operational with the new config
• In parallel, I will start working on getting the configuration for the Registry container ready

When we finish that, we plan to have the following sequence:

1. Get the registry on separate nodes in production
2. Work on getting the staging registry running in containers

added meta label

changed milestone to %WoW ending 2017-07-04

removed milestone

mentioned in issue peopleops#311 (closed)

I'm creating the production infrastructure here: https://gitlab.com/gitlab-com/infrastructure/issues/2181

The production infrastructure is ready and we'll switch to it in https://gitlab.com/gitlab-com/infrastructure/issues/2188.

Registry is running on its own fleet in production!

Should we close this issue and open a new one for the containerisation?

@omame we already have that in https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2441, shall we reuse that?

The scope of that one is about the final goal. We can keep updating it but splitting the work needed to get there in smaller issues here.

@omame I think we should close this one as it is done, and open the next one which will be about containerizing the registry and running it with an orchestrator.

changed milestone to %WoW ending 2017-07-11

closed

mentioned in issue #2205

added cloud native label

mentioned in issue organization#106