Environment-specific variables

Description

Developers and Ops should be able to specify different values for secret (and not-so-secret) variables, and tie them to environments. For example, doing a deploy to Heroku might need an app name, which you could specify by $HEROKU_APP_NAME, but that name would be different in staging and production.

Proposal

• Add a scope to variables so that you could say var A applies to environment B.
• Support wildcards, so you could say that some deploy variable applies to review/* and all review apps would get the variable automatically.
• Default scope is *.

Links

• #17986 (closed)
• #17633 (closed)

mentioned in issue #14698 (moved)

As discussed yesterday with @sytses, we need something like this to make sure that developers pushing to side branches are unable to deploy to production. We now have a security problem.

Our use case:

• Developers push to side branches
• Developers make a Merge request to master
• When code is reviewed and tested, the Merge request is merged
• The code is deployed using Gitlab CI and Secret Variables the developers can not view or edit: .gitlab-ci.yml has a deployment Job with

only:
  - master

However, the developer could change .gitlab-ci.yml

only:
  - name-of-sidebranch

Since Secret Variables are available on all side branches, this will deploy the application.

mentioned in issue #20800 (closed)

mentioned in issue #20805 (closed)

I think it should be useful to have ability to specify Secure Environment Variables per Environment especially for deployments.

mentioned in issue #20870 (closed)

When I was exploring "environments" today, this feature was the first thing I was expecting there. Without it I don't really see use of the environments.

The second thing I was expecting was to use .gitlab-ci.yml only as a template and then each environment would be instance of such template, a new job to run.

Example: I have an application running on staging server and multiple production servers, each for another customer. It does not make sense to create and commit a job in .gitlab-ci.yml for each customer. Each production installation has a SSH key and URL where to deploy, but otherwise the deploy is always the same. So when creating installation to a new customer, I would expect to define a new environment, specify secret variable with ssh key, not-so-secret variable with deploy destination and a task which implements the deploy using environment-specific variables. Then Gitlab CI would kick in and install new version to production server every time the specified branch is updated. Which branch it is should be specified in the environment configuration (customer may have dedicated branch).

mentioned in issue #21277 (closed)

mentioned in issue #21911 (moved)

mentioned in merge request !6275 (merged)

Mentioned in issue #22190 (closed)

@markpundsack maybe schedule this for 8.13?

Milestone changed to %8.13

Yes, this aligns well with the needs in #22190 (closed), so tentatively scheduling for 8.13.

/cc @ayufan

Environment-specific variables should take https://gitlab.com/gitlab-org/gitlab-ce/issues/21971#convention-over-configuration into account. One option is to add a scope to variables so that you could say var A applies to environment B. But by including wildcards, you could say that some deploy variable applies to review/* and all review apps would get the variable automatically.

Mentioned in issue #22445 (closed)

Mentioned in issue #21770 (moved)

I'm not sure whether #21770 (moved) is worth considering in this (perhaps a consolidated approach?), it discusses global/per-person variables, which might relate to your scope comment.

Added ~113220 label

Customer requesting this and related security from some of the above-linked-tickets here: https://gitlab.zendesk.com/agent/tickets/40980

I think this would probably cover our environment use cases and would probably take care of per-user/global CI variables:

1. Production
2. Stage
3. QA
4. Single Developer branch/environment + multiple developer branches/environments (idea here is that developers push a branch for a feature and an environment get spun up for them for that feature)

I think this should also have permissions attached to those environment level permissions so that at least production/master environment variables can be heavily locked down.

Mentioned in issue #22904 (closed)

We're currently looking at moving from Jenkins CI -> GitLab Ci for our primary test suite. A feature we use a lot of in the Jenkins world is the ability for each of the jenkins build projects to environment variables defined in isolation of other other projects. This means that our developers can add flags to their test builds and we have separate master builds based on different configurations of flags. It's important to note that we currently depend on being able to run builds against the same branch with different environment variables.

Is this something that is likely to be addressed? Is it part of this issue? Should I raise another issue? I'm also willing to try and contribute towards its development, as I'd love to get us off of jenkins.

Milestone changed to %8.14

@BageDevimo Not sure exactly what your need is, but that's not likely to be addressed by this issue. Please create a new issue with your use-case.

Mentioned in issue #23492 (closed)

Milestone changed to %Backlog

Mentioned in issue #23861 (moved)

Mentioned in issue #21358 (closed)

Mentioned in issue #21911 (moved)

Will this issue be addressed soon?
It's hard for me to estimate, but if the complexity of this feature is not too high i'd be glad to try and contribute.

I'd gladly welcome the feature since I have many variables tied to environment. For instance, I need to define each build script for each environment while they're doing the same thing, only because they use different variables values:

res=$(curl -X PUT --data-binary @deploy/webdeploy.zip -u "$AZURE_PUBLISH_NAME_DEVELOP:$AZURE_PUBLISH_PASSWORD_DEVELOP" --silent --output /dev/null --write-out %{http_code} "$AZURE_PUBLISH_URL_DEVELOP")

mentioned in issue #28274 (moved)

+1

added ~1071781 label

added ~423634 label

mentioned in issue #29475 (closed)

I could use this for deploying to rancher. Each rancher environment requires a different access-key/secret-key. And it knows which environment you are interacting with based on the access key. So being able to set this on the environment would be a huge improvement.

mentioned in issue #27722 (closed)

mentioned in issue #25813 (moved)

mentioned in issue #20368 (closed)

added ~123454 label

Concerning myself, this is the missing piece preventing me to use Gitlab as a complete CI/CD workflow. I think Secret Management per environment should be reprioritize and integrated to Gitlab Product.

mentioned in issue #17739 (moved)

Current thinking is to use a single list for variables, but add a field for which environment(s) the variables should be available in, and this field would support wildcards.

If we implement #24196 (closed) in GitLab CE, we'll have basic variable protection for all. We could then consider making this issue EE only as it's a somewhat more advanced requirement.

added ci variables label

mentioned in issue #31296 (closed)

mentioned in issue #20261 (closed)

changed milestone to %9.4

removed milestone

mentioned in issue #38316

mentioned in issue gitlab#6077

mentioned in issue gitlab#6785