Using Gitlab for CI/CD on Docker-swarm

What is your audience?

This is for DevOps users with intermediate experience which wants to see how to quickly and easily use GitLab-ci to deploy on a Docker Swarm Cluster

What are the requirements?

• a GitLab account
• a valid Play-With-Docker session

What's your ETA?

This is the hard question ;) I hope several days only since today (08/08/2017)

---

Proposal

Writing sample

This is an exemple tutorial I Submit on Play-with-docker about using Docker Swarm to deploy micro-service apps:

• Using Traefik Load Balancer
• Using Docker Flow Proxy

In this proposal I would like to show you how to use a Docker Swarm within your GitLab Repository for Pipelines and Environments

Introduction

Outlines

If we comes their that means that we have our project on Gitlab, and we wants to automatically deploy it on a Docker Swarm cluster.

For that purpose, I will show you how to create a Docker Swarm cluster using Play-With-Docker. Note, that this method is only for you to ease to follow this tutorial, but the Play-with-docker cluster will only be available for 4 hours, afterwards it will be discarded. So If you have a Swarm Cluster on your own It would be better to uses yours.

Example project

This will be this one https://gitlab.com/allamand/example-voting-app/

The goal it to build, test and deploy to a docker swarm the docker voting app example. The deployment will be done on Play-with-docker instances, where we set-up a One docker swarm instance, deploy Traefik loadbalancer, in order to proxify requests to our voting application, then deploy the voting app.

Let me know If this sounds correct for you.

Thanks @allamand! Seems interesting. Would you please detail?

Key questions to keep in mind:

• What is "it"?
• Which problems does it solve?
• Why is this interesting?
• How do I get there?
• What tools do I need to make it work?

Thank you! :)

Hi @marcia,

I'm hesitating between a customer storie or a tutorial on how we actually use gitlab to deploy each branch in a fully testable live environment on either a Docker Swarm backend or a CloudFoundry backend (maybe it's need to separate post, one for each ?).

The problem is solves is to accelerate project delivery time and increase quality of code : each git push will be unitary tested, then gitlab will create a fully testable environment in which we could set-up more than just 1 app and make end-to-end integration test.

It is interesting for us because it shows a use case how to benefit from tracability (GitLab logs everything), automation for each steps of deployment, all in only one place : GitLab

Tools needed : GitLab instance, a Docker Swarm cluster for deployment on it, a CloudFoundry environment for deployment on it, gitlab runners with API access to either Docker Swarm and CloudFoundry.

What do you think ?

added blog-posts~1268475 label

@allamand Looking forward to see your post, very interesting

Did that ever materialise? We are considering using Gitlab CI with a Swarm as well, but it doesn't seem to support it out of the box.

Hi @batjko @paoloyx, didn't have time to work on it until now but I can found some now. @marcia are you interested ? I could try to demo this using a swarm with play-with-docker

+1 for this - going through deploying this way now (using docker swarm and secret variables with Gitlab CI) this tutorial would have been awesome to have - mainly connecting to a remote swarm and deploying to it

are you interested? I could try to demo this using a swarm with play-with-docker

Yes, @allamand we're interested in this tutorial :) Though please advise, we're changing the rules of the Community Writers program; I'll link them from here as soon as they're live. What you could do now is making your demo work first, then, if you're okay with the new rules, we'll be able to move forward.

added blog-posts~1828473 label

moved from blog-posts#326 (closed)

@allamand update:

The Community Writers Program has changed. Here's the new webpage and the new terms and conditions. If you're happy with them, feel free to submit your proposal according to the new steps.

In that case, please update your issue's description according to this issue template.

added Waiting for reply label

@allamand Any news of your proposal Sebastien ?

I guess there is still lot of people who are interested over there.

changed the description

Thanks for your comment @o314!

@allamand what's your ETA?

changed the description

@marcia, I've just update the description and I have a project demo which can be use as base (not quite finished). But I'm out of office next 2 weeks, so a reasonable ETA should be 31/08 ?

Hi @marcia, How can I proceed to send you a draft ?

@allamand It's all here: https://about.gitlab.com/handbook/product/technical-writing/community-writers/terms-and-conditions/#steps-to-getting-published

added up-for-grabs and removed Waiting for reply labels

@marcia I would like to write about this subject and I accept the terms of the Community Writers Program.

changed the description

I wanted to update the introduction in the issue but I've got an error for now while trying to edit..

added Pre-assess label

assigned to @marcia

@allamand thank you! Here's my general feedback over the content you added to this issue's description:

-1. It's too much for an introduction, you need to be more concise than that. ;)

-2. From what you defined as your audience:

This is for DevOps users with intermediate experience which wants to see how to quickly and easily use GitLab-ci to deploy on a Docker Swarm Cluster

And your requirements:

It may be a GitLab account, and a Docker Swarm available, and flows open from a gitlab-runner instance to the docker swarm.

I understand this is a tutorial, or a user guide, not a technical overview. Correct? ref If so, you should of course, briefly introduce the subject, then go right over the how-to.

If you have no docker swarm we will try to use one on Play-With-Docker

-3. What do you mean by "try"?

assigned to @allamand

unassigned @marcia

added Waiting for reply label

I wanted to update the introduction in the issue but I've got an error for now while trying to edit..

@allamand this might have happened when we were deploying a new patch release or smt like that. Could you please try again? If it doesn't work, I'd ask you to please open a new issue in CE explaining what is happening, then ping me there :)

changed time estimate to 3d

added GitLab CI label

@marcia, yes it is a tutorial with step-by-step to follow. I'll concise the introduction meanwhile you can find the draft here

changed the description

meanwhile you can find the draft here

@allamand thanks for the link, but this draft is technically published and available publicly. If you want to submit it to the community writers program, please take the content offline.

Community Writers Program - terms and conditions - requirements:

Content must be original, comprehensible, technical, and unprecedented.

Please read "original and unprecedented" as unpublished. ;)

@allamand from your draft, the general feedback is: it's overwhelming, we can't accept it as is. You need to focus on your article's main subject: "how to use a Docker Swarm with GitLab CI/CD and multiple environments". Adjust the title for a How-to, and make your content a very straight tutorial:

• short intro
• step 1
• step 2
• step...
• conclusion

Please reduce it to at least 50% of your draft, then ping me back.

I'm looking forward to see your new version!

Thanks!

@marcia, sorry for mistake, i've unpublished the content. How will I send you new version ?

@marcia, sorry for mistake, i've unpublished the content.

No worries, thank you. :)

How will I send you new version?

Let's do it step by step. I took a look at your draft because you linked from here, and I thought would be important to let you know it was too long. But let's focus on the intro and the outlines first.

Please take your time to have a second look at the steps to getting published. It's essential that you are familiar with the process.

I know you're having trouble updating this issue's description, so please add them (intro and outlines) in comments to this thread.

@marcia, Please find below the intro, and the toc (still have problem to edit the issue)

Introduction

GitLab offers very powerful to create Continuous Integration & Deployment Pipelines for each of your product/projects. The Goal is to have code frequently pushed, built, tested and deployed, in an automated way you previously defined.

In this tutorial I will show you how to automatically benefits from such a pipeline to build an Application composed of several Containerized service with Docker, and to deploy it to a Docker Swarm container orchestrator that will be in charge of spawning and scale the containers, dispatching theme on the servers and activate network communications. The same approach can be used to deploy to others orchestration services such as Kubernetes or CloudFoundry.

GitLab manage our project application source code, and we leverage GitLab CI to manage our Continuous Delivery pipeline. We define a single YAML file which describe the process in which we can create an unlimited number of jobs that will be executed by GitLab.com public Runners (but you could also easilly setup your own runners).

With this step-by-step tutorial I will show you how to uses GitLab.com to :

• Build and test an application (based on docker sample repo : docker voting example application )
• Create an on the fly docker swarm cluster on Play with Docker platform
• Deploy the application on the docker swarm.

Table of Contents

• How to use a Docker Swarm with GitLab CI/CD
  • Introduction
  • Step 1: Create GitLab project for our application
    • Project description
  • Step 2: Manually testing the Deployment (optional)
    • Step 2.1: Create a Docker Swarm Cluster on Play-with-Docker
    • Step 2.2: Deploy Traefik Reverse Proxy on Docker Swarm
    • Step 2.3: Build & Deploy our Application on Docker Swarm
  • Step 3: Automate Deployment using Gitlab CI Pipeline
    • Step 3.1: Fork the application
    • Step 3.2: Configure GitLab with Play-with-Docker URL
    • Step 3.3: Trigger your first Pipeline
    • Step 3.4: Explaining the pipeline
      • image and services
      • stages
      • Job1: Build a Push to Registry
      • Job2: Pull & deploy & test locally
      • Job3: Deploy to Swarm
    • Step 3.5 Validating the deployment
  • Conclusion

Hi @marcia, Does the content comply with your guidelines ? have a nice day.

removed Waiting for reply label

Thanks @allamand, sorry for not having replied earlier, we were getting GitLab 10.0 ready for y'all!

Does the content comply with your guidelines ?

Yes it does, thank you :)

I asked for my colleague's second opinion on this. @axil will help me to better evaluate your proposal. Pls hang tight till then.

assigned to @axil

Quick question, the title of this issue here mentions Cloud Foundry, but I didn't find any reference about it in the tutorial.

I'll leave a comment about the tutorial in https://gitlab.com/gitlab-com/community-writers/issues/157 :)

Thanks @axil!

@allamand I'll be on vacation, Axil will conduct the evaluation of your article and help you till I'm back. :)

@axil, the first proposal was to cover both swarm & cloudfoundry, but there is too much content, so I only deal with docker swarm in this one (and anyone can reproduce the tutorial), it will be more difficult with cloudfoundry I don't know if it exists some kind of play-with-cloudfoundry to help following the tutorial

changed title from Using Gitlab for CI/CD on Docker-swarm & Cloud Foundry Environments to Using Gitlab for CI/CD on Docker-swarm

Cool, I updated the title :)