Ability to protect gitlab-ci.yml from changing by developers

Added ~19173 feature proposal labels

@gsmetal Thanks for the suggestion I think that we already have an issue for that somewhere.

Really, I spent batch of time in searching, but found nothing related

Thought about it again and decided that make this option global for project is more correct, because with same .gitlab-ci.yml runner will do same work on any branch so pushing to unprotected branch will still cause unwanted jobs to run.

I am going to implement this in the near future by doing something like this:

Add boolean attribute allow_developers_to_change_ci_config to projects
Add checkbox somewhere near only_allow_merge_if_build_succeeds, only if builds enabled
Add check somewhere in Checks::ChangeAccess
Disable edit of .gitlab-ci.yml from web-interface

I didn't fully dive in code yet, but I think something like this will work.

IMHO the best way to address this would be through out of tree .gitlab-ci.yml as per #15041 (moved).

If someone modifies .gitlab-ci.yml actively and use strategies like feature-branching, they wouldn't want to have single .gitlab-ci.yml file. They need different versions of it.

IMHO the only way to do safe changes and still keep ability to have different versions in different branches — "protection" flag for yml file.

Yeah, there's another issue for this, but with a slightly different proposal, in that they want a warning when looking at a MR that contains a change to the .gitlab-ci.yml.

Probably this is a duplicate of #20461 (moved)

mentioned in issue #20461 (moved)

the main problem, that it is not enough to concentrate only on yml.

Because you deploy not with yml, but with some script that corresponds to the yml job. So it is enough to change this job, not yml itself. Even more - it will be hard for you to understand in mr this. Actually problem is actual. but solution doesn't look very obvious.

@lingvokot This may be next step in protecting CI configuration. For example, add ability to specify number of files (paths) that is CI configuration in addition to .gitlab-ci.yml and deny their change by developers too unless allow_developers_to_change_ci_config is true.

@markpundsack Sorry for mentioning, but I don't know better way to ask some questions about development. I'm trying to implement what I told before and realized, that now you don't have methods to work with commits body (changed files, message etc.) in pre-receive checks, but you do have them in Gitlab EE. Is it possible to move some functions you use in push_rules feature (this for example) from EE and what is the best way to do this? I need this for my solution, but I don't want to reimplement such methods as they already implemented in different place.

Yes, protecting just .gitlab-ci.yml isn't really enough. You'd have to protect any related scripts as well. But then the malicious develop could always zero-out the tests themselves. So do you block changes to tests? That doesn't really work because developers are supposed to write new tests. So do you somehow enforce that only new tests are added but no tests are changed or removed? I mean, that sounds kinda cool, and if this is really important, perhaps it's something that's covered by something a little more intelligent. Perhaps something like Danger?

But, isn't this what the whole code review process is for? With EE, you can protect master, enforce a review flow, and block developers from merging anything that hasn't both been approved and passed CI. Shouldn't someone approving the MR see if the developer has dome something malicious?

Putting a checkbox to protect the .gitlab-ci.yml feels like a placebo. It makes you feel like you're protecting something, but not actually providing any real protection.

@gsmetal Now that you mention it, if this kind of feature were to ship, it really should be part of EE, not CE. This isn't something a small team (should) care about. This is something enterprises that care about enforcing policies (over trusting humans) wants.

But in terms of automating things to make it easier for reviewers, I believe Danger has worked on GitLab integration. You might want to take a look at what it can do.

Yes, protecting just .gitlab-ci.yml isn't really enough, to protect test environments. But it is enough, to protect deploy process (or some kind of specific job, with dedicated runner).

For example, you could:

Add jobs that tests your project and assign them to runner with tag test. That runner could be secured some how by using docker, or be unsecure at all.
Add job that deploys your project and assign it to runner with tag deploy. That job could use some commands or some data (private keys), that is present only on that runner.

So, to protect deploy runner from developers , we could allow changing tags in .gitlab-ci.yml (or whole .gitlab-ci.yml) only by masters.

@gsmetal Now that you mention it, if this kind of feature were to ship, it really should be part of EE, not CE. This isn't something a small team (should) care about. This is something enterprises that care about enforcing policies (over trusting humans) wants.

@markpundsack, IMHO many small teams do not give access to production infrastructure to all their developers. And IMHO this is really missing feature for deployments, pipelines, etc.

P.S. To complete the workflow, we need also "Allow Developers to add tags?" and/or "Allow Developers to do manual job approval?".

I agree with @distol . As an example - small 5 people team like (3 dev, lead, designer) - wants to restrict deployment only for lead and one dev.

mentioned in issue #20261 (closed)

Yes, protecting just .gitlab-ci.yml isn't really enough, to protect test environments. But it is enough, to protect deploy process (or some kind of specific job, with dedicated runner).

True! I hadn't thought about the tags and dedicated runner case.

mentioned in issue #21911 (moved)

Mentioned in issue #21911 (moved)

Everyone is in agreement that this would be a useful feature to protect the deployment process. Whats required for this feature to progress further the acceptance ladder?

If someone is interested I've written small patch to make this available "just now". It restricts pushing .gitlab-ci.yml to only members with Master or higher permissions. You can apply it by patch -p1 < protect_ci.patch from root gitlab directory (/opt/gitlab/embedded/service/gitlab-rails for omnibus). It is working for us on production for several days.

I started working on MR for this (as seen here before), but then thought that it's needed more complex approach for this and related problems and more decisions for developers to make, so we stopped at this small patch without touching databases, UI and making configurations.

Mentioned in issue #24794 (moved)

We've found another way to achieve security of CI configuration changes. Please, review #24794 (moved).

I've updated patch, it had problems with pushing merged changes, that already were pushed to another branch. And now it restrics changes only to owners and supports true in ci_admin column if it exists.

Developers should be able to change .gitlab-ci.yml in a feature branch I think. We should secure environment variables to masters only, not scripts.

added ~1071781 ~113220 labels

@sytses @markpundsack I think this issue is targeting the wrong problem. From my POV, I don't care if people change the .gitlab-ci.yml file as long as I have force push on master disabled as that will force a review and encourages the right behavior.

I do care if someone can access a protected runner or production secrets in their feature branch. So locking down runners/secrets to specific branches may solve this in a better manner without killing the ability of developers to propose changes to CI. It may make it harder to test things, but I don't see a better way of doing it.

@sidewinder12s I agree. Protecting variables and runners is the better approach and is higher priority for us. That said, I know some people still want to protect .gitlab-ci.yml itself; some for the wrong reasons, but possibly some for a good reason. I'm reluctant to reject this proposal outright, at least until we've delivered the other variable/runner pieces and proven that the need is resolved. But as of now, I have no intention of moving forward with this issue. /cc @cpallares

My current plans are noted in https://gitlab.com/gitlab-org/gitlab-ce/issues/21911#note_27088752.

added blocked label

I might be going about this the wrong way but I was considering adding a runner to my public repository that had IAM permissions (likely through a role but potentially could be set in ~/.aws/credentials) and tagging it as production.

Then I could have the production deploy run on the production tagged runner and have that only on master pushes.

Unfortunately because it is a public repository that then means that anyone can raise a merge/pull request against the repo and modify the .gitlab-ci.yml file to run anything they want against the master tagged runner just by setting the tags against their stage. This could then lead to a leak of credentials or just straight up executing whatever they want against the AWS account that the runner has access to.

Obviously I can restrict IAM permissions to very fine grained access but if I need the ability to update something then there's nothing stopping someone replacing my production service with whatever they want.

If there's a better way of restricting how a build can be ran against a runner then that would be good to hear about too.

@tomelliff I think the alternative would be to include your ssh key for the production machine as a protected variable available only on the master branch, and then ssh into the production machine, rather than running gitlab-ci runner on it directly.

But I might be totally wrong with this...

Currently you would need to protect master and require an approval/merge from yourself or someone else in a higher level of authority to approve all merges into master and watch for someone unauthorized editing the gitlab-ci.yml file.

@sidewinder12s any push will trigger a CI pipeline, isnit?

The solution we have found (copyright Jules B. ^^) to really secure C.D. (via rsync + ssh) is to use Docker:

The Docker image contains the ssh key
The Docker image has a script that cannot be edited by developer
The script checks that the "GITLAB_USER_ID" can deploy
The script deploys the project A to production servers matching A only

Okay, my $0.02. I think a solution where you would restrict Secret Variables usage to one or multiple branches would be best. Because the underlying issue here is that a developer can obtain access to the production environment, either by changing the script being run there (and when, for example "always" instead of only on master) or by obtaining the credentials themself by printing them. This solution allows everybody to propose changes to deploy scripts with Merge Requests, and everybody can review them.

Also, currently one could solve this issue by using a file lock on the .gitlab-ci.yml.

With the release of protected variables in 9.3 I think something equivalent but for protected runners would be great for solving this.

If I have a deployment runner that is locked to a project (so that forks can't access it) and is only used for deployments (using tags) then making sure that it will only ever run on a protected branch prevents someone with developer access changing the .gitlab-ci.yml file so that they can deploy on any branch, circumventing controls.

This allows me to have secure runners that have necessary permissions/network access to be able to make changes to my environments.

I'd guess this could be exposed in the runner configuration with something like a --protected flag or protected: true in the .toml file.

This approach (rather than protecting the .gitlab-ci.yml file) has the added benefit that developers are free to change the pipeline tests as they see fit but are unable to force changes through without master access presuming you only allow masters to merge to your protected branch(es).

mentioned in issue #12729 (closed)

Here's another usecase, I wonder if it would be possible with the current plans (for #20826 (moved) and #15041 (moved)):

I would like to host my friend's static website and I would like to give him a repository in my GitLab where he can put his files. I would create a .gitlab-ci.yml with a tag for the machine where his website should be put and a copy (or docker build) script.

He's not very tech-savvy so let's assume he doesn't protect his keys well.

Right now I can't deploy with Gitlab-CI because whoever gains access to his key can change .gitlab-ci.yml and change the destination directory of the copy script or even change the tags to take over a completely different runner.

mentioned in issue #15041 (moved)

moved to gitlab#5784

Hey!

GitLab is moving all development for both GitLab Community Edition and Enterprise Edition into a single codebase. The current gitlab-ce repository will become a read-only mirror, without any proprietary code. All development is moved to the current gitlab-ee repository, which we will rename to just gitlab in the coming weeks. As part of this migration, issues will be moved to the current gitlab-ee project.

If you have any questions about all of this, please ask them in our dedicated FAQ issue.

https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

You can also find more information here:

https://gitlab.com/gitlab-org/gitlab-ee/issues/13304
https://about.gitlab.com/2019/08/23/a-single-codebase-for-gitlab-community-and-enterprise-edition/

Frequently Asked Questions

For an up to date list of questions and answers, please take a look at https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

What will we do with the repository names?

https://gitlab.com/gitlab-org/gitlab-ee/ will become https://gitlab.com/gitlab-org/gitlab, and https://gitlab.com/gitlab-org/gitlab-ce will become https://gitlab.com/gitlab-org/gitlab-foss.

Why rename gitlab-ce to gitlab-foss?

Using "gitlab" and "gitlab-ce" would be confusing, so we decided to rename gitlab-ce to gitlab-foss to make the purpose of this FOSS repository more clear

I created a merge requests for CE, and this got closed. What do I need to do?

You will need to create a merge request at https://gitlab.com/gitlab-org/gitlab-ee/. This link will eventually redirect to https://gitlab.com/gitlab-org/gitlab.

How does the licensing work in this new setup?

Everything in the ee/ directory is proprietary. Everything else is free and open source software. If your merge request does not change anything in the ee/ directory, the process of contributing changes is the same as when using the gitlab-ce repository.

Will you accept merge requests on the gitlab-ce/gitlab-foss project after it has been renamed?

No. Merge requests submitted to this project will be closed automatically.

Will I still be able to view old issues and merge requests in gitlab-ce/gitlab-foss?

Yes.

How will this affect users of GitLab CE using Omnibus?

No changes will be necessary, as the packages built remain the same.

How will this affect users of GitLab CE that build from source?

Once the project has been renamed, you will need to change your Git remotes to use this new URL. GitLab will take care of redirecting Git operations so there is no hard deadline, but we recommend doing this as soon as the projects have been renamed.

Where can I see a timeline of the remaining steps?

https://gitlab.com/gitlab-org/gitlab-ee/issues/13304

Will community contributions submitted to the new "gitlab" repository be available in the new gitlab-foss repository?

Yes, these changes will be synced to gitlab-foss automatically, several times per day.

locked this issue

mentioned in issue gitlab#6127

mentioned in issue gitlab#7137

Ability to protect gitlab-ci.yml from changing by developers

Description

Proposal

Designs

Child items ...

Activity

Frequently Asked Questions

Admin message

Admin message

Ability to protect gitlab-ci.yml from changing by developers

Description

Proposal

Activity

Frequently Asked Questions