Auto Deploy should use project's helm chart, if available

Description

It's awesome that Auto Deploy deploys a Docker image automatically for a project, but the configuration of services and ingress is hard-coded. We should let developers customize how their apps are deployed, yet still take advantage of the rest Auto Deploy has to offer.

Pecking order:

• If project has Helm chart (./chart/Chart.yaml) or has variable pointing to a chart (AUTO_DEPLOY_CHART), use it
• If project has docker-compose-bundle.dab, use it to derive k8s config (using Kompose?) (https://gitlab.com/gitlab-org/gitlab-ce/issues/37588)
• If project has docker-compose.yml, use it to derive k8s config (using Kompose?) (https://gitlab.com/gitlab-org/gitlab-ce/issues/37588)
• If project has Dockerfile, use it, with default k8s config (ideal from Helm chart, but currently from parametrized YML)
• If nothing, use Herokuish buildpacks to create Docker image, and default k8s config to deploy

Going further, ideally Auto Deploy would add a default helm chart to your project's repo for you, so you have a place to start for edits and incremental learning.

Proposal

Links / references

• May build on https://gitlab.com/gitlab-org/gitlab-ce/issues/30723 option 2.
• Auto deploy supports docker compose files via Kompose: https://gitlab.com/gitlab-org/gitlab-ce/issues/29003

Documentation blurb

Auto Deploy deploys a Docker image automatically for a project, including automatic configuration of a good CI/CD pipeline. But the configuration of services and ingress is hard-coded and if you want something custom, you have to give up the rest of the benefits of Auto Deploy.

GitLab 9.4 adds support for leveraging Helm Charts when using Auto Deploy. Simply create a project-level variable CI_AUTO_DEPLOY_CHART to point to a Helm chart, this will now be used for deployment rather than the built-in chart. The value can be a reference to a stable chart, a full URL for another chart, or even a filesystem path to a chart embedded in your project repo.

mentioned in issue #30723 (closed)

added Stretch label

changed milestone to %9.2

Thinking more about this, perhaps we could let people specify a URL to a helm chart rather than requiring them to include the files in their project repo. In the variable, they could still point to a directory in their repo if they want (e.g. ./chart), but this lets them point to an external URL instead, or even a stable helm chart. We won't likely support charts in private repos this way though, but perhaps that could be a future iteration if their charts are in other GitLab projects.

/cc @ayufan @sytses

mentioned in issue gitlab-examples/kubernetes-deploy#7 (closed)

mentioned in issue #30748 (closed)

Bundling a chart with the project isn't a good idea. Individual containers are building blocks of charts, there isn't a one-to-one relationship, and dynamic application configuration shouldn't be stored with the application (it defeats the purpose). Charts should be stored separately and supported in the CI/CD pipeline.

mentioned in issue #29412 (closed)

@ashernevins Yes, those are all great, valid points. Yet despite that, a lot of developers choose the experience of having the chart in the same repo. Maybe it only works for "simple" configurations. Or maybe for systems with multiple services, there's still a project responsible for deploying them all, and the helm chart could be in that project.

I also think we can move some of that dynamic configuration to project variables, such as scale (number of replicas). And then some of the deploy configuration kinda fits in the repo, if you squint just right.

I have wondered if we should have some kind of official adjust repo, just for deploy configuration. Kind of like wikis and snippets can be associated with a project, and are still version controlled, but outside of the project's main repo. Just a thought.

mentioned in issue #29003 (moved)

From simple checks it seems that we can freely use Helm now with external repository.

added docs-missing label

@sytses suggested having a chart keyword in .gitlab-ci.yml, just like we have image keywords. It's overkill at this point, especially because Auto Deploy is not a first-class construct; it's just a template. But something to consider for the future.

For the record, I'm not in favor of adding a chart keyword at this time. The simplest iteration is to use an environment variable and there's not enough value in a dedicated keyword.

mentioned in issue #32638 (moved)

@ayufan This slipped from 9.2 as a Stretch; can we make it Deliverable for 9.3?

mentioned in issue #32829 (closed)

added Deliverable ~901060 and removed Stretch labels

changed milestone to %9.4

mentioned in merge request gitlab-ci-yml!80 (closed)

changed milestone to %9.5

Quote extracted from a kubernetes sig-apps thread about CI/CD that may be interrest you:

One result I’d personally like to see is the workflow management detached from the templating/package format.

As a user, I shouldn’t care if the resources are packaged using opencompose, ksonnet, helm, or ‘plain’ kubernetes resources. I would like to manage/deploy/upgrade my applications in a consistent way regardless of the templating format, using my favorite tool (e.g. kubectl, Helm-Tiller, an operator, kpm) and not having to switch between them depending on the app maintainer preferences.

One of our goals for the next kubernetes release cycle is to create a kubectl plugin, part of which can explore this idea:

$ kubectl appr deploy quay.io/ant31/cookieapp 
   -> deploying from a 'docker-compose.yml'  (kompose)
$ kubectl appr deploy quay.io/charts/gitlab 
   -> deploying from 'helm-chart' package
$ kubectl appr deploy quay.io/ksonnet/redis 
   -> deploying from 'ksonnet' package
$ kubectl appr deploy quay.io/k8s/kubernetes-dashboard 
   -> deploying from 'plain kubernetes resources’

marked this issue as related to #32829 (closed)

changed milestone to %10.0

assigned to @ayufan and @nolith

added In review label

mentioned in merge request gitlab-ci-yml!88 (merged)

https://gitlab.com/gitlab-org/gitlab-ci-yml/merge_requests/88 implements all the requirements except docker-compose support.

Variable name is not the same as discussed in https://gitlab.com/gitlab-org/gitlab-ci-yml/merge_requests/88#note_39367911

mentioned in issue #37588

changed the description

Moving Docker Compose to future issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/37588

Closed by https://gitlab.com/gitlab-org/gitlab-ci-yml/merge_requests/88

closed