Make auto deploy more practical

Review feature from a user perspective, detect potential problems, propose improvements.

Related to https://gitlab.com/gitlab-org/gitlab-ce/issues/32829 and https://gitlab.com/gitlab-org/gitlab-ce/issues/32638

cc @markpundsack

added ~19173 auto devops labels

I played with autodeploy and its great to see that some simple syntax like that could work:

image: registry.gitlab.com/gitlab-examples/kubernetes-deploy

build:
  stage: build
  script:
    - command build
  only:
    - master

deploy:
  stage: deploy
  script:
    - command deploy
  environment:
    name: production
    url: http://k1.dzaporozhets.me
  variables:
    DISABLE_POSTGRES: "yes"
  only:
    - master

However, I struggled a lot with such trivial task as deploying existing Ruby application from GitLab.com to my Google Container Engine. Basically, there was no single document or guide that covers this use case. As result, I collected necessary information from many different pages, example projects, and guides.

The first improvement I have in my mind that should increase feature adoption is to write a guide "Simple way to deploy your GitLab.com application to Google Container Engine". It should contain next resources:

1. Actual step-by-step instruction including creating cluster and filling Kubernetes integration page on GitLab.com project setting page.
2. Link to the public project dedicated to this example. So I can fork this project, fill my Kubernetes settings and it will be deployed to my cloud. Preferably without a single modification.

ToC could look like that:

1. Fork our GitLab.com Ruby project
2. Create cluster on Google Container Engine.
3. Connect to cluster and copy permissions to Project -> Settings -> Kubernetes
4. Push a change to GitLab.com project
5. Open IP address of your deployment. Wow, GitLab actually deployed an app to my Cloud.
6. Explain content of .gitlab-ci.yml file. What happens in build and deploy sections

@markpundsack what do you think?

@dzaporozhets That sounds like a good start. Note, we're in the process of updating the template significantly (https://gitlab.com/gitlab-org/gitlab-ci-yml/merge_requests/80) but the base ease of use should still be there.

This sounds like a good technical article, or tutorial, to put in the docs. Currently the best guide is the I2P demo (https://about.gitlab.com/handbook/product/i2p-demo), but it starts from GitLab running in a cluster, rather than GitLab.com.

Currently the best guide is the I2P demo (https://about.gitlab.com/handbook/product/i2p-demo), but it starts from GitLab running in a cluster, rather than GitLab.com.

yep thats was a problem for me. If I already use GitLab.com or run existing instance it would be hard to enable the feature without good guide.

@ayufan @markpundsack I was using auto deploy for Ruby application https://gitlab.com/dzaporozhets/sample-ruby-app and everything works great. But I have one question though. Our auto deploy feature creates service and ingress for my app but I don't know what is a preferable and easy way to expose app to the world.

cc @bikebilly

I started a merge request with a guide how to use Auto Deploy with GitLab.com project and Google cloud https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2413

@JobV is it possible now to measure how many projects use Auto Deploy on GitLab.com? For example by using our deploy image ( https://gitlab.com/gitlab-examples/kubernetes-deploy )

The guide is merged. TODO: write a short blog post with a link to the guide.

@dzaporozhets I'm sure there is some query we could do. Maybe @ayufan or @grzesiek could help with that.

mentioned in merge request gitlab-com/www-gitlab-com!6965 (merged)

@grzesiek It would be great to get an indication of Auto Deploy usage, and checking gitlab.com for projects with .gitlab-ci.yml referencing kubernetes-deploy is the best proposal I've seen.

@markpundsack What do you mean by "referencing kubernetes-deploy"? Do you mean using image: registry.gitlab.com/gitlab-examples/kubernetes-deploy, thus having jobs using this image in the database?

@grzesiek Yes.

@markpundsack @dzaporozhets I guess that the most simple way is to check ci_builds rows that reference kubernetes-deploy in options, maybe something like

SELECT COUNT(DISTINCT project_id) FROM ci_builds WHERE options LIKE '%kubernetes-deploy%';

I have neither production, nor staging access, so I can't check it - we probably need to open an issue in infrastructure issue tracker. This query can take a lot of time on production, not sure if there is more efficient way.

@grzesiek what if we check project has active integration with Kubernetes? Something like

SELECT COUNT(*) FROM "services" WHERE "services"."type" IN ('KubernetesService') AND "services"."active" = true

It should be fast and number should be close to accurate.

@dzaporozhets This might be enough indeed, but I suppose that most of the projects with kubernetes service enabled do not use auto deploy feature, but I may be wrong and I'm curious what @markpundsack thinks.

@grzesiek I created an issue https://gitlab.com/gitlab-com/infrastructure/issues/2433 to run both. So we can compare data

Results of command is here

Summary:

• 528 projects with Kubernetes integration enabled
• 147 projects with kubernetes-deploy

TODO: Do same queries in September and October.

In meantime blog post should go public in few days

moved to gitlab#10445

Hey!

GitLab is moving all development for both GitLab Community Edition and Enterprise Edition into a single codebase. The current gitlab-ce repository will become a read-only mirror, without any proprietary code. All development is moved to the current gitlab-ee repository, which we will rename to just gitlab in the coming weeks. As part of this migration, issues will be moved to the current gitlab-ee project.

If you have any questions about all of this, please ask them in our dedicated FAQ issue.

https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

You can also find more information here:

• https://gitlab.com/gitlab-org/gitlab-ee/issues/13304
• https://about.gitlab.com/2019/08/23/a-single-codebase-for-gitlab-community-and-enterprise-edition/

Frequently Asked Questions

For an up to date list of questions and answers, please take a look at https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

What will we do with the repository names?

https://gitlab.com/gitlab-org/gitlab-ee/ will become https://gitlab.com/gitlab-org/gitlab, and https://gitlab.com/gitlab-org/gitlab-ce will become https://gitlab.com/gitlab-org/gitlab-foss.

Why rename gitlab-ce to gitlab-foss?

Using "gitlab" and "gitlab-ce" would be confusing, so we decided to rename gitlab-ce to gitlab-foss to make the purpose of this FOSS repository more clear

I created a merge requests for CE, and this got closed. What do I need to do?

You will need to create a merge request at https://gitlab.com/gitlab-org/gitlab-ee/. This link will eventually redirect to https://gitlab.com/gitlab-org/gitlab.

How does the licensing work in this new setup?

Everything in the ee/ directory is proprietary. Everything else is free and open source software. If your merge request does not change anything in the ee/ directory, the process of contributing changes is the same as when using the gitlab-ce repository.

Will you accept merge requests on the gitlab-ce/gitlab-foss project after it has been renamed?

No. Merge requests submitted to this project will be closed automatically.

Will I still be able to view old issues and merge requests in gitlab-ce/gitlab-foss?

Yes.

How will this affect users of GitLab CE using Omnibus?

No changes will be necessary, as the packages built remain the same.

How will this affect users of GitLab CE that build from source?

Once the project has been renamed, you will need to change your Git remotes to use this new URL. GitLab will take care of redirecting Git operations so there is no hard deadline, but we recommend doing this as soon as the projects have been renamed.

Where can I see a timeline of the remaining steps?

https://gitlab.com/gitlab-org/gitlab-ee/issues/13304

Will community contributions submitted to the new "gitlab" repository be available in the new gitlab-foss repository?

Yes, these changes will be synced to gitlab-foss automatically, several times per day.

locked this issue