common-ci-tasks
This project includes reuseable GitLab-CI jobs for the GitLab SaaS Platforms team.
Setting up a new project? Use the project template.
Developer Setup
Looking to setup your workstation? Follow the documentation at: docs/developer-setup.md
.
Warning about Refs
It is possible to use these templates without a ref
tag when including them in your project, but this is not recommended as it may break your build process and make it non-repeatable. For example, if you revert a change, the templates will still reference the latest template which may no longer be compatible. For this reason, it's recommended you always use refs for your templates.
Dockerhub image override
All templates that use an upstream image from dockerhub (e.g anything without a hostname as part of the image path) by default are prefixed with docker.io
. This can be changed (for example to use GitLab's dependency proxy or GCR pull through cache) when you include the templates by providing an input value for docker_hub_host
like so (this example uses dependency proxy):
- project: "gitlab-com/gl-infra/common-ci-tasks"
ref: v2.8.0
file: templates/standard.yml
inputs:
docker_hub_host: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}
Templates
It is preferable to use templates over individual CI jobs as it reduces clutter, increases standardization and improves consistency across projects. Additionally, new functionality can be added to projects with only an upgrade to the ref
.
Template Include | Description |
---|---|
templates/standard.yml |
This template should be included in all Infrastructure projects. It includes standard checks, gitlab-scanners, validations and release processes common to all projects using this template library. |
templates/golang.yml |
This template should be used for Golang projects. |
templates/terraform.yml |
This template should be used for Terraform projects. It performs standard validity checks against the Terraform files in the project. |
CI Jobs
Alternatively to using the templates above, you can include the following individual jobs.
Include | Description |
---|---|
gitlab-scanners.yml |
This adds various GitLab SAST, Dependency Scanner, Secret Detection, Licence Scanning, Container Scanning and IAC Scanner tools. |
asdf-tool-versions |
Checks that the .tool-versions file is synced with .gitlab-ci-asdf-versions.yml, that all the plugins are declared in ./scripts/install-asdf-plugins.sh and that ASDF is generally working. |
editorconfig-check.yml |
This job ensures that the repository adheres to the EditorConfig rules for the project. |
go-mod-tidy.yml |
Ensures that go mod tidy is up-to-date and go.mod and go.sum are tidy. |
go-unittests.yml |
Runs unit tests to ensure and emits a JUnit XML report for GitLab. |
golangci-lint.yml |
Runs golangci-lint on the project. |
goreleaser.yml |
Including this template will build a binary release of the project using GoReleaser. |
renovate-bot.yml |
Runs renovatebot against the project to automatically upgrade dependencies. |
semantic-release.yml |
This job will run semantic-release on your repository. This allows the repository to be automatically tagged based on conventional commit messages. |
shellcheck.yml |
Performs linting for shell scripts using Shellcheck. |
shfmt.yml |
Validates that shell-scripts use a canonical formatting. |
terraform-format.yml |
Runs terraform fmt to ensure that all Terraform files are correctly formatted. |
terraform-validate.yml |
Runs terraform validate to ensure that all Terraform files are valid. |
tflint.yml |
Runs tflint across all directories that contain *.tf files. |
Project Template
common-ci-tasks
favours convention-over-configuration, and expects configuration files and some directories to be configured in a certain way. The quickest way to get going on a new project is to use the common-template-copier
project template to generate a new project based on the conventions expected by these CI tasks.
More information on using project templates can be found in the project templates documentation..
Developing Templates
Testing
The project https://gitlab.com/gitlab-com/gl-infra/common-ci-task-tests is a test-bed for the tasks in this project.
When a template is changed, a downstream CI/CD pipeline will be initiated in the common-ci-task-tests
project,
running the current branch of common-ci-task-tests
.
This provides some basic testing of functionality. Over time, these unit tests should be expanded to include more edge-cases and more templates.