Proper private project handling for Auto DevOps
Description
We have partial support for private project handling in Auto Deploys, but there are serious limitations. We need a proper solution for production-grade usage.
Proposal
Links / references
Documentation blurb
(Write the start of the documentation of this feature here, include:
- Why should someone use it; what's the underlying problem.
- What is the solution.
- How does someone use this
During implementation, this can then be copied and used as a starter for the documentation.)
Activity
mentioned in issue #32638 (moved)
changed milestone to %9.5
mentioned in issue #32829 (closed)
mentioned in issue #34524 (moved)
@bikebilly Auto Deploy creates a k8s secret with
CI_JOB_TOKENso that k8s can fetch images from GitLab's container registry. Unfortunately that token expires after the pipeline job finishes, so the image can’t be fetched on pod restarts, etc. Currently, you need to use a personal access token instead to have persistent image fetching. We need some easy mechanism that Auto Deploy can make use of that results in a persistent ability for k8s to re-fetch the image.One potential solution is to automatically create a persistent project-level token, and pass that token in via
CI_REGISTRY_PASSWORD. See https://gitlab.com/gitlab-org/gitlab-ce/issues/34524.added Stretch label
marked this issue as related to #32829 (closed)
removed Stretch label
changed milestone to %Backlog
changed milestone to %10.1
added Deliverable label
See https://gitlab.com/charts/charts.gitlab.io/issues/81 for possible fix, or at least improvement.
marked this issue as related to charts/charts.gitlab.io#81
As @ayufan pointed out, using
IfNotPresentworks well, until you add a node. I just verified that adding a node and trying to scale does fail withFailed to pull image.If that's truly the only circumstance, then we might be able to add a more explicit warning, and someone may choose to use Auto DevOps in production if they know they have a fixed cluster size (forever). I imagine the images may be lost if nodes are upgraded though, so it's still a pretty heavy restriction.
@markpundsack Almost true, it will work as long as you do run the app again on the same node as it was run in the past. So if you run one replica, the image will be downloaded only once, on one of the nodes. So, this is hard from optimal. Also, if you don't run image on the node for some time the image will be garbage collected, thus will disappear :)
We just ran into this. There's no way we could work with those constraints in production or even heavy dev use. Our nodes sometimes have pods evicted for various reasons that have nothing to do with new deploys of the apps: resource constraints, node maintenance, et cetera. The secret has to work for as long as the app is deployed.
Right now we're doing this with a user and a personal access token, but that takes up a seat license unnecessarily. And we can't use any real user, as that user leaving the org shouldn't break everything. Worse, we have to create a user with visibility into all projects.
The idea of a project-level set of registry access credentials sounds very nice.
mentioned in issue #38366
moved to gitlab#9524
Hey!
GitLab is moving all development for both GitLab Community Edition and Enterprise Edition into a single codebase. The current
gitlab-cerepository will become a read-only mirror, without any proprietary code. All development is moved to the currentgitlab-eerepository, which we will rename to justgitlabin the coming weeks. As part of this migration, issues will be moved to the currentgitlab-eeproject.If you have any questions about all of this, please ask them in our dedicated FAQ issue.
https://gitlab.com/gitlab-org/gitlab-ee/issues/13855You can also find more information here:
https://gitlab.com/gitlab-org/gitlab-ee/issues/13304- https://about.gitlab.com/2019/08/23/a-single-codebase-for-gitlab-community-and-enterprise-edition/
Frequently Asked Questions
For an up to date list of questions and answers, please take a look at
https://gitlab.com/gitlab-org/gitlab-ee/issues/13855What will we do with the repository names?
https://gitlab.com/gitlab-org/gitlab-ee/ will become https://gitlab.com/gitlab-org/gitlab, and https://gitlab.com/gitlab-org/gitlab-ce will become https://gitlab.com/gitlab-org/gitlab-foss.
Why rename gitlab-ce to gitlab-foss?
Using "gitlab" and "gitlab-ce" would be confusing, so we decided to rename gitlab-ce to gitlab-foss to make the purpose of this FOSS repository more clear
I created a merge requests for CE, and this got closed. What do I need to do?
You will need to create a merge request at https://gitlab.com/gitlab-org/gitlab-ee/. This link will eventually redirect to https://gitlab.com/gitlab-org/gitlab.
How does the licensing work in this new setup?
Everything in the
ee/directory is proprietary. Everything else is free and open source software. If your merge request does not change anything in theee/directory, the process of contributing changes is the same as when using the gitlab-ce repository.Will you accept merge requests on the gitlab-ce/gitlab-foss project after it has been renamed?
No. Merge requests submitted to this project will be closed automatically.
Will I still be able to view old issues and merge requests in gitlab-ce/gitlab-foss?
Yes.
How will this affect users of GitLab CE using Omnibus?
No changes will be necessary, as the packages built remain the same.
How will this affect users of GitLab CE that build from source?
Once the project has been renamed, you will need to change your Git remotes to use this new URL. GitLab will take care of redirecting Git operations so there is no hard deadline, but we recommend doing this as soon as the projects have been renamed.
Where can I see a timeline of the remaining steps?
https://gitlab.com/gitlab-org/gitlab-ee/issues/13304Will community contributions submitted to the new "gitlab" repository be available in the new gitlab-foss repository?
Yes, these changes will be synced to gitlab-foss automatically, several times per day.
