Speed up CI pipeline

What does this MR do?

• Introduces a image that is used as a base for all other jobs. Image is built only if the branch name starts with docker-ci-image/, in other cases it's just used by a pipeline. All time-consuming tasks (like dependencies installation with apt-get install or gem install or go get) were moved to this image's build definition.
• Parallelizes build and package stages to make them faster.

Why was this MR needed?

To speed up test and build process :)

Are there points in the code the reviewer needs to double check?

Does this MR meet the acceptance criteria?

• Documentation created/updated
• Tests
  • Added for this feature/bug
  • All builds are passing
• Branch has no merge conflicts with master (if you do - rebase it please)

What are the relevant issue numbers?

added 1 commit

• f498de15 - Prepare custom dockerfile to use with CI jobs

Compare with previous version

added 1 commit

• a50b8fab - Prepare custom dockerfile to use with CI jobs

Compare with previous version

added 1 commit

• f8ff33c5 - Prepare custom dockerfile to use with CI jobs

Compare with previous version

added 1 commit

• e44cda1d - Prepare custom dockerfile to use with CI jobs

Compare with previous version

added 1 commit

• 57cd7e9a - Prepare custom dockerfile to use with CI jobs

Compare with previous version

added 1 commit

• 0a05236e - Prepare custom dockerfile to use with CI jobs

Compare with previous version

added 1 commit

• 685f5b66 - Prepare custom dockerfile to use with CI jobs

Compare with previous version

added 1 commit

• fd1d7670 - Prepare custom dockerfile to use with CI jobs

Compare with previous version

added 1 commit

• 97c0d5cf - Pararelize binaries build stage

Compare with previous version

changed title from [WIP] Prepare custom dockerfile to use with CI jobs to [WIP] Speed up CI pipeline

added 1 commit

• b2f1133d - Paralelize packages stage

Compare with previous version

added 1 commit

• f6293d4a - Paralelize packages stage

Compare with previous version

added 1 commit

• f6293d4a - Paralelize packages stage

Compare with previous version

added 2 commits

• 35e5ab7e - Parallelize build stage
• 0a5d1da2 - Parallelize packages stage

Compare with previous version

unmarked as a Work In Progress

@ayufan Can you review this?

assigned to @ayufan

I'm not sure exactly what is happening here, as there's a lot of changes.

assigned to @tmaczukin

With current version of code and Registry enabled for gitlab-org/gitlab-ci-multi-runner we will automatically start publishing images in our registry. Do we wan't this? The only drawback I see is the space usage when most of people will still use version downloaded from Docker Hub.

If we decide that we don't want to publish Runner and helper images in our registry at this moment, then I need also to adjust ci/release_docker_images script.

Edit: If we don't want to publish images in our registry we also need to update ci/release_docker_images on 1-10-stable, 1-11-stable and 9-0-stable branches.

resolved all discussions

added 12 commits

• 0a5d1da2...b10bcc72 - 8 commits from branch master
• b0500604 - Prepare custom dockerfile to use with CI jobs
• a92f7799 - Parallelize build stage
• 8a4fa170 - Parallelize packages stage
• e9a4cf0a - Version 'ci' image with golang version

Compare with previous version

approved this merge request

merged

mentioned in issue #1220 (closed)