gitlab-runner merge requestshttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests2017-10-06T11:00:50Zhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/513Mention tagged S3 sources in installation documentation2017-10-06T11:00:50ZTomasz Maczukintomasz@gitlab.comMention tagged S3 sources in installation documentation## What does this MR do?
Mention tagged S3 sources in installation documentation. Currently it is described only in Bleeding Edge installation docs which may be not read by people who want to install GitLab Runner in different way. Wi...## What does this MR do?
Mention tagged S3 sources in installation documentation. Currently it is described only in Bleeding Edge installation docs which may be not read by people who want to install GitLab Runner in different way. With this MR documentation for each OS with manual installation steps will have a link to description how to download a binary/package for a specific version of Runner.
This MR also makes steps in `linux-manually.md` numbered.
References #2192v1.10Achilleas PipinellisAchilleas Pipinellishttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/443Add support for GIT_SUBMODULE_STRATEGY2017-08-30T13:37:52Zusername-removed-90962Add support for GIT_SUBMODULE_STRATEGY## What does this MR do?
Adds support for a `GIT_SUBMODULE_STRATEGY` variable which enables initializing submodules during the "get sources" phase.
## Why was this MR needed?
See the discussion here: https://gitlab.com/gitlab-org/gitl...## What does this MR do?
Adds support for a `GIT_SUBMODULE_STRATEGY` variable which enables initializing submodules during the "get sources" phase.
## Why was this MR needed?
See the discussion here: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/101#note_18965117.
Git submodules are a very common way of handling external code. Submodules are [officially supported ](https://docs.gitlab.com/ce/ci/git_submodules.html), but require manually adding `git submodule update --init` in the `before_script` of `.gitlab-ci.yml`. For the `docker` executor, this means the user-specified `image` must have `git` installed (which is the whole reason the `gitlab/gitlab-runner-helper` images exist).
Additionally, automatic certificate injection does not happen. This means that with GitLab using a self-signed cert (or cert from a custom CA), one must manually inject the CA certificate in the `before_script`.
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [x] Documentation created/updated
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Closes #101v1.10https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/518Limit number of concurrent requests to builds/register.json2017-08-01T10:14:49ZKamil TrzcińśkiLimit number of concurrent requests to builds/register.jsonThis makes `builds/register` to run with limit. Currently, due to long processing we can queue a lot of requests at single time which increases impact on GitLab.com. This makes number of requests to be run at single time to be 1. General...This makes `builds/register` to run with limit. Currently, due to long processing we can queue a lot of requests at single time which increases impact on GitLab.com. This makes number of requests to be run at single time to be 1. Generally we should extend that to all other updates, but currently this is good enough.
I would seriously consider to backport that to at least `v1.11.x`.
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?v1.10Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/529Fix runner crashing on builds helper collect2017-04-28T09:51:16ZTomasz Maczukintomasz@gitlab.comFix runner crashing on builds helper collect## What does this MR do?
Prevents Runner from raising `panic` when `BuildsHelper` metrics are collected.
## Why was this MR needed?
Please read #2228 for a reference
## Are there points in the code the reviewer needs to doubl...## What does this MR do?
Prevents Runner from raising `panic` when `BuildsHelper` metrics are collected.
## Why was this MR needed?
Please read #2228 for a reference
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Closes #2228v1.10Nick ThomasNick Thomashttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/530Replace b.CurrentStage with b.CurrentState where it was misused2017-04-04T18:20:41ZTomasz Maczukintomasz@gitlab.comReplace b.CurrentStage with b.CurrentState where it was misused## What does this MR do?
Replace `b.CurrentStage` with `b.CurrentState` in `run()` in `common/build.go`. Fields are misused here.
## Why was this MR needed?
Please read #2274 for a reference
## Are there points in the code the review...## What does this MR do?
Replace `b.CurrentStage` with `b.CurrentState` in `run()` in `common/build.go`. Fields are misused here.
## Why was this MR needed?
Please read #2274 for a reference
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?v1.10https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/526Don't install docs for the fpm Gem2017-03-30T10:51:38ZZeger-Jan van de Wegzegerjan@gitlab.comDon't install docs for the fpm Gemv1.10https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/391Add configuration options for kubernetss resource requests2017-03-20T14:20:30Zusername-removed-357684Add configuration options for kubernetss resource requests## What does this MR do?
Adds configuration options for container resource requirements (http://kubernetes.io/docs/user-guide/compute-resources/) similar to how resource limits are implemented.
## Why was this MR needed?
Setting res...## What does this MR do?
Adds configuration options for container resource requirements (http://kubernetes.io/docs/user-guide/compute-resources/) similar to how resource limits are implemented.
## Why was this MR needed?
Setting resource limits will throttle builds, even if the node running the build pod has free resources.
With resource requests one can tell the kubernetes scheduler the minimum requirements for a container/pod without actually limiting it's resource consumption (i.e. faster builds when node/cluster is idle).
## Are there points in the code the reviewer needs to double check?
I've prefixed the currently used options for limits (`cpus`, `memory`) with "limits_" to help distinguish limits and requests. This is a breaking change so maybe a more clever solution is needed.
I'm happy to update docs when we're sure about the option names.
## Does this MR meet the acceptance criteria?
- [x] Documentation created/updated
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Closes #2019 v1.10Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/463Fix race in helpers/prometheus/log_hook.go: Fire() method2017-01-31T14:24:42ZTomasz Maczukintomasz@gitlab.comFix race in helpers/prometheus/log_hook.go: Fire() method## What does this MR do?
Fixes races that can occur with concurrent calls to Fire() in `helpers/prometheus/log_hook.go`.
## Why was this MR needed?
https://gitlab.com/gitlab-com/infrastructure/issues/1079#note_22260206
## Are there p...## What does this MR do?
Fixes races that can occur with concurrent calls to Fire() in `helpers/prometheus/log_hook.go`.
## Why was this MR needed?
https://gitlab.com/gitlab-com/infrastructure/issues/1079#note_22260206
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [x] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?v1.10Kamil TrzcińśkiKamil Trzcińśkihttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/461Fix crash on machine creation2017-01-27T12:59:56ZKamil TrzcińśkiFix crash on machine creationThis MR fixes a critical failure encountered in https://gitlab.com/gitlab-com/infrastructure/issues/1073
@tmaczukin Could you release a new patch release ASAP?This MR fixes a critical failure encountered in https://gitlab.com/gitlab-com/infrastructure/issues/1073
@tmaczukin Could you release a new patch release ASAP?v1.10Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/456Reduce size of gitlab-runner-helper images2017-01-25T23:30:31Zusername-removed-90962Reduce size of gitlab-runner-helper images## What does this MR do?
This MR reduces the size of the `gitlab-runner-helper` images by:
- Using `miniperl` instead of `perl` (required for `git-submodule`)
- Using `apk --no-cache` when installing Alpine packages
```
REPOSITORY ...## What does this MR do?
This MR reduces the size of the `gitlab-runner-helper` images by:
- Using `miniperl` instead of `perl` (required for `git-submodule`)
- Using `apk --no-cache` when installing Alpine packages
```
REPOSITORY TAG IMAGE ID CREATED SIZE
gitlab/gitlab-runner-helper arm-7f95086 a6c261cb3c57 5 days ago 135.7 MB
gitlab/gitlab-runner-helper x86_64-7f95086 150caa5f8496 5 days ago 81.79 MB
gitlab/gitlab-runner-helper arm-d18f38e 5c70f6e2a381 29 minutes ago 101.7 MB
gitlab/gitlab-runner-helper x86_64-d18f38e 539912ee8c70 30 minutes ago 40.38 MB
```
- The `arm` image size is reduced by 34MB (25.0%)
- The `x86_64` image size is reduced by 41.4MB (50.6%)
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
See https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/443/diffs#ad79990b3e834afa27f538ac9b0bf4e57f84f7c3_4_3
cc @ayufan @tmaczukinv1.10https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/455Fix S3 and packagecloud uploads step in release process2017-01-23T20:01:10ZTomasz Maczukintomasz@gitlab.comFix S3 and packagecloud uploads step in release process## What does this MR do?
Fixes the error when artifacts uploaded to S3 have bigger size than the default `--max-size` value for travis-ci/artifacts tool. Also it splits S3 and packagecloud uploads to two separate builds, so it'll be eas...## What does this MR do?
Fixes the error when artifacts uploaded to S3 have bigger size than the default `--max-size` value for travis-ci/artifacts tool. Also it splits S3 and packagecloud uploads to two separate builds, so it'll be easier to retry a failed build in some cases.
## Why was this MR needed?
To improve process of releasing artifacts to S3 and packagecloud.
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Closes #2072v1.10Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/438Add '--run-untagged=false' cli option for runners2017-01-23T07:26:23Zusername-removed-630712Add '--run-untagged=false' cli option for runners> This is a general Merge Request template. Consider to choose a template
> from the list above if it will match your case more.
Provides support in the runner to register with api parameter 'run_untagged=false' by omitting the boole...> This is a general Merge Request template. Consider to choose a template
> from the list above if it will match your case more.
Provides support in the runner to register with api parameter 'run_untagged=false' by omitting the boolean cli flag `--run-untagged`;
`false` has also been made the default for the case when a tag list is specified, so that a runner that registers with a tag list will automatically
send `run_untagged=false` in the registration request, unless the `--run-untagged` boolean was specified.
Implements support for #1732
v1.10https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/444Kubernetes Namespace Overwrite2017-01-22T00:07:27ZTomasz Maczukintomasz@gitlab.comKubernetes Namespace Overwrite## What does this MR do?
Successor of !389.
This MR gives more flexibility for CI pipelines based on Kubernetes Runner. It adds a environment variable to overwrite the target Kubernetes namespace. Therefore, you can for instance, use H...## What does this MR do?
Successor of !389.
This MR gives more flexibility for CI pipelines based on Kubernetes Runner. It adds a environment variable to overwrite the target Kubernetes namespace. Therefore, you can for instance, use Helm/Landscaper to install Charts and create the set of services/pods that are required to execute the CI pipeline step, and also the ability to clean-up by deleting an namespace completely.
The namespace can only be overwritten if matches a configured regular expression (added to the runner configuration), to guarantee only designed namespace names can be employed. When this configuration is empty, this whole overwrite behavior is disabled.
## Why was this MR needed?
To create more flexible CI runs, to include `Helm` packages in a brand new namespace which will be deleted altogether afterwards. Using an overwrite mechanism avoid an extra steps to achieve the same goal.
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [x] Documentation created/updated
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?v1.10Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/453Docker Executor: "Preparation failed: no such image" is no longer mysterious2017-01-21T21:02:19Zusername-removed-536451nowaker@virtkick.comDocker Executor: "Preparation failed: no such image" is no longer mysterious## What does this MR do? & Why was this MR needed?
Previously it wasn't clear what image is missing. That was caused by the fact the code didn't print service name and version *before* raising error. Now it's clear.
## Are there points...## What does this MR do? & Why was this MR needed?
Previously it wasn't clear what image is missing. That was caused by the fact the code didn't print service name and version *before* raising error. Now it's clear.
## Are there points in the code the reviewer needs to double check?
No.
## Does this MR meet the acceptance criteria?
I guess these all are N/A due to simplicity of this contribution.
- [N/A] Documentation created/updated
- Tests
- [N/A] Added for this feature/bug
- [x] All builds are passing
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Fixes #2028. CC @tmaczukin.v1.10https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/449Pass ImagePullSecrets2017-01-21T20:43:04ZKamil TrzcińśkiPass ImagePullSecrets> This is a general Merge Request template. Consider to choose a template
> from the list above if it will match your case more.
## What does this MR do?
It adds support for `ImagePullSecrets` which allows to use private docker registri...> This is a general Merge Request template. Consider to choose a template
> from the list above if it will match your case more.
## What does this MR do?
It adds support for `ImagePullSecrets` which allows to use private docker registries.
Refer to this documentation: https://kubernetes.io/docs/user-guide/images/
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?v1.10Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/445Exclude unused options from AbstractExecutor.Build.Options2017-01-21T04:14:32ZTomasz Maczukintomasz@gitlab.comExclude unused options from AbstractExecutor.Build.Options## What does this MR do?
Adds a mechanism to exclude unused keys from `options` map sent with build response.
## Why was this MR needed?
Some options, for example `environment` are serialized by GitLab to the `options` map, but ...## What does this MR do?
Adds a mechanism to exclude unused keys from `options` map sent with build response.
## Why was this MR needed?
Some options, for example `environment` are serialized by GitLab to the `options` map, but are not used by Runner at all. And since Runner is verifying received options against options supported by used executor and shell this ends with an annoying warning like described in #1706.
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Closes #1706v1.10Kamil TrzcińśkiKamil Trzcińśkihttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/439Add Prometheus metric that counts number of catched errors2017-01-21T03:33:19ZTomasz Maczukintomasz@gitlab.comAdd Prometheus metric that counts number of catched errors## What does this MR do?
Adds a new Prometheus metric that counts the number of errors. Counting is done by `logrus` hook and it generates metrics like:
```
# HELP ci_runner_errors The number of catched errors.
# TYPE ci_runner_errors...## What does this MR do?
Adds a new Prometheus metric that counts the number of errors. Counting is done by `logrus` hook and it generates metrics like:
```
# HELP ci_runner_errors The number of catched errors.
# TYPE ci_runner_errors counter
ci_runner_errors{level="error"} 0
ci_runner_errors{level="fatal"} 0
ci_runner_errors{level="panic"} 0
ci_runner_errors{level="warning"} 7
```
## Why was this MR needed?
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Closes #2041v1.10Kamil TrzcińśkiKamil Trzcińśkihttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/452Create index file for S3 downloads2017-01-21T03:20:44ZTomasz Maczukintomasz@gitlab.comCreate index file for S3 downloads## What does this MR do?
Adds the step in release scripts that creates an `index.html` file with links to all files uploaded with a release to S3.
## Why was this MR needed?
## Are there points in the code the reviewer needs to double...## What does this MR do?
Adds the step in release scripts that creates an `index.html` file with links to all files uploaded with a release to S3.
## Why was this MR needed?
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?v1.10Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/383Kubernetes termination grace period2017-01-17T08:39:03Zusername-removed-357684Kubernetes termination grace period## What does this MR do?
Add a configuration option for the termination grace period of build pods.
## Why was this MR needed?
When a build is finished (e.g. Pod is deleted) Kubernetes will send SIGTERM to applications in the containers...## What does this MR do?
Add a configuration option for the termination grace period of build pods.
## Why was this MR needed?
When a build is finished (e.g. Pod is deleted) Kubernetes will send SIGTERM to applications in the containers of the Pod, which can be handled in order to effect graceful termination. SIGKILL is sent a configurable number of seconds later if the application does not terminate sooner (defaults to 30 seconds, controlled by `spec.terminationGracePeriodSeconds`).
Lots of fast, parallel builds will leave Pods of finished builds behind for 30 seconds, blocking cluster resources. With a lower (or even `0`) `terminationGracePeriodSeconds`, Pods of finished builds will be cleaned up immediately.
## Are there points in the code the reviewer needs to double check?
@munnerz could you give me a hint on how to integrate that into the existing tests?
## Does this MR meet the acceptance criteria?
- [x] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [x] All builds are passing
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?v1.10https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/384Add poll interval and timeout parameters for Kubernetes runner2017-01-17T08:39:03Zusername-removed-838134Add poll interval and timeout parameters for Kubernetes runner## What does this MR do?
This MR adds two new parameters that can be specified in the `[runners.kubernetes]` configuration:
* `poll_interval`: How frequently, in seconds, the runner will poll the Kubernetes cluster to check on the sta...## What does this MR do?
This MR adds two new parameters that can be specified in the `[runners.kubernetes]` configuration:
* `poll_interval`: How frequently, in seconds, the runner will poll the Kubernetes cluster to check on the status of the container it has attempted to create. [Default: 3]
* `poll_timeout`: The amount of time, in seconds, that needs to pass before the runner will timeout attempting to connect to the container it has just tried to create. [Default: 180]
## Why was this MR needed?
These parameters help with the use case whereby you have a Kubernetes cluster that has potential for high workload, and there isn't enough resources to run every task as soon as they come in (i.e. the Kubernetes scheduler could be reporting that that pod hasn't started because there isn't enough resources to run the container). Currently, there are some magic constants that define that the runner will poll Kubernetes every 3 seconds, at most 60 times, before timing out, but it allows for more functionality if these values can be configured.
This is not the perfect solution for this situation; ideally GitLab CI would be able to talk to the Kubernetes scheduler when it was about to run a Kubernetes task, and hold off on actually scheduling the task if it knew there wasn't resources currently available. However, this seems like a much more complex undertaking, and may go against the GitLab CI flow quite a bit (but if people think this is possible in future, I can open an issue for it).
In the meantime, the addition of these parameters should help those in use cases like myself where there's a potential for backlog due to full resources, and don't want the tasks timing out too early.
## Does this MR meet the acceptance criteria?
- [x] Documentation created/updated
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)v1.10Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.com