Production should use "production" build artifacts

One of the build team's core goals for 2017 is to provide production quality methods for deploying GitLab in a cloud native way. One of the largest cloud deployments of GitLab in the cloud is actually ourselves, with GitLab.com.

While we should ensure that our deployment methods are extremely easy to use and get started with, they should also be flexible enough to be able to scale up to GitLab.com scale. This means our customers can be assured our standard supported build tools can scale as high as they need, and production will be able to utilize production deployment assets instead of inventing their own.

To ensure this is possible, our production GitLab.com service should utilize the same deployment methods we recommend to customers. If it doesn't work for GitLab.com, we should strongly consider whether we are doing something wrong.

Some of the existing projects we want to accomplish in 2017 to enable this:

• Provide a container per service (https://gitlab.com/charts/charts.gitlab.io/issues/14)
• Postgres HA https://gitlab.com/gitlab-org/omnibus-gitlab/issues/1807

What else do we need to consider to allow this to happen?

• I imagine one of the challenges is that we are not leveraging a scheduler in production. This means while we strongly recommend using Kubernetes to our customers, we are not following this model for GitLab.com.
• This means we depend on other tools to provide these missing features, like Consul or Terraform.
• Our logging and security stack, I believe based on ELK, is not something we typically provision for customers.

changed the description

@pcarranza can we use this as a basis to discuss? I'm the build PM and so would really like to both ensure that what we provide customers is of high quality and can scale.. and that we can drink our own wine and not have to have duplicate sets of deployment assets.

If GitLab.com can't use what we make, then we should discuss why and how to get aligned.

mentioned in issue #1504

@joshlambert I think this is great and I agree that this is what we need to do.

From the tip of my head, and as a conversation opener, one of the main issues we have found with the current package is its monolithic nature, this drove us to analyze how the current production infrastructure looks like, and you can see in this graph how everything is connected to everything, making it all an extremely tangled mess.

I think that the first step we need to take is to first understand how far the package (or containers) go - do they include a logging service? do they include a production class monitoring setup? - and then we can talk about what bits are useful for me.

Another thing to consider is: if we are going the helm charts way and we will be leveraging the community provided images like Postgresql or Prometheus, then it doesn't make sense to make a huge fuzz about changing what we have now for something that GitLab will be using from the community itself.

In other words, why don't we start with scoping?

From my perspective what I would like is to first disassemble the current monolith in multiple independent services, then we can play with the jigsaw to see how things fit together and see what are we missing.

@joshlambert

One thing to mention though is that the subject of this issue is Production should use "production" build artifacts.

Production is using the production build artifacts currently

The problem is that the current production build artifact is hard to use in production at our scale because of its monolith nature.

@joshlambert

This means we depend on other tools to provide these missing features, like Consul or Terraform.

I don't think that this is a missing feature of GitLab the package. This is an infrastructure decision.

I would like to understand exactly where are you coming from because this kind of thinking is showing me that there is a lack of understanding of what those tools do and why are they in use.

cc/ @ernstvn BTW

@joshlambert, @pcarranza, great topic, and one that has been receiving thought from a few angles. Josh, can you schedule a call for late this week / early next so we can accelerate the process of getting on the same page? \cc @stanhu

@ernstvn we have one scheduled already with @marin for this Friday, let me know if you want invites too.

I'd like an invite as this is an issue near and dear to my that I've been campaigning for half a year for .

@marin, @pcarranza , yes please :-)

added meta label

This issue, as it is written down and has been discussed already is not really actionable per se, and a bit too wide to fit in any reasonable iteration.

I think it's safe to close this issue as the discussion around cloud native is already ongoing.

closed