gitlab-runner merge requestshttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests2017-02-23T20:16:49Zhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/492cleanup windows install docs2017-02-23T20:16:49Zusername-removed-5332cleanup windows install docs## What does this MR do?
- Add missing steps to Windows install details
- Use `gitlab-runner` rather than `gitlab-ci-multi-runner.exe` in support of https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1195
- Document ...## What does this MR do?
- Add missing steps to Windows install details
- Use `gitlab-runner` rather than `gitlab-ci-multi-runner.exe` in support of https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1195
- Document how to use with true service account (Built-in System Account)
- General cleanup
- Provide better link on how to run elevated command prompt
Replaces https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/482https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/482cleanup windows install docs2017-02-23T20:10:20Zusername-removed-5332cleanup windows install docs## What does this MR do?
* Add missing steps to Windows install details
* Use `gitlab-runner` rather than `gitlab-ci-multi-runner.exe` in support of https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1195
* Document how to...## What does this MR do?
* Add missing steps to Windows install details
* Use `gitlab-runner` rather than `gitlab-ci-multi-runner.exe` in support of https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1195
* Document how to use with true service account (Built-in System Account)
* General cleanup
* Provide better link on how to run elevated command prompthttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/471Resolve ""x509: certificate signed by unknown authority" error when running job"2017-02-10T03:43:02Zusername-removed-1091253Resolve ""x509: certificate signed by unknown authority" error when running job"> This is a general Merge Request template. Consider to choose a template
> from the list above if it will match your case more.
## What does this MR do?
## Why was this MR needed?
## Are there points in the code the reviewer needs to...> This is a general Merge Request template. Consider to choose a template
> from the list above if it will match your case more.
## What does this MR do?
## Why was this MR needed?
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Closes #1https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/469Added --cpus option support.2017-05-25T13:33:42Zusername-removed-15146Added --cpus option support.## What does this MR do?
Adds CPUS option for docker runner.
## Why was this MR needed?
Adds support for the --cpus option available in Docker 1.13. This allows limit number of CPUs available to each docker container.
## Are ...## What does this MR do?
Adds CPUS option for docker runner.
## Why was this MR needed?
Adds support for the --cpus option available in Docker 1.13. This allows limit number of CPUs available to each docker container.
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [x] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
* #2101 -- Support Docker 1.13 --cpus option9.3https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/467Set GO15VENDOREXPERIMENT2017-02-07T15:31:20ZZeger-Jan van de Wegzegerjan@gitlab.comSet GO15VENDOREXPERIMENTNick ThomasNick Thomashttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/441WIP: add proxy documentation2017-10-06T11:01:47Zusername-removed-25949WIP: add proxy documentation## What does this MR do?
This MR adds the documentation on how to run the runner behind a proxy.
/cc @axil First attempt for adding this to the docs. Is this the right place?
I added it as a seperate file so we can link to it fr...## What does this MR do?
This MR adds the documentation on how to run the runner behind a proxy.
/cc @axil First attempt for adding this to the docs. Is this the right place?
I added it as a seperate file so we can link to it from `linux-repository.md` and `linux-manual.md`
I'm pretty sure this'll need a few iterations before the wording is satisfatory. Can you please do a first review?
Closes https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1537Achilleas PipinellisAchilleas Pipinellishttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/432[WIP] Tempoprary add 'force' option to NetworkConnectionOptions2017-03-06T23:56:49ZTomasz Maczukintomasz@gitlab.com[WIP] Tempoprary add 'force' option to NetworkConnectionOptions## What does this MR do?
Adds a `Force` option to `NetworkConnectionOptions` struct to force stale containers disconnection from networks (which extends the code added in !390.
This is the temporary fix to test if such workflow wil...## What does this MR do?
Adds a `Force` option to `NetworkConnectionOptions` struct to force stale containers disconnection from networks (which extends the code added in !390.
This is the temporary fix to test if such workflow will resolve problems described in #1642. It should be fixed properly with switching to `github.com/docker/docker/client` (!301 - waiting for Docker 1.13 to be released).
## Why was this MR needed?
Please read #1642 for a reference.
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Related to #1642v9.0Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/431WIP: Google Cloud Storage cache server support2017-01-06T02:27:18Zusername-removed-587507WIP: Google Cloud Storage cache server support> This is a general Merge Request template. Consider to choose a template
> from the list above if it will match your case more.
## What does this MR do?
## Why was this MR needed?
## Are there points in the code the reviewer needs to...> This is a general Merge Request template. Consider to choose a template
> from the list above if it will match your case more.
## What does this MR do?
## Why was this MR needed?
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Closes #1773https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/417Adds Path and Share settings to S3 cache2016-12-21T21:33:51Zusername-removed-707439Adds Path and Share settings to S3 cache## What does this MR do?
Adds settings `Shared` and `Path` to cache settings. Both will modify resulting URL as pointed out by @ayufan in #1897:
* By default, we keep current behavior, URLs have the form `/runner/<token>/project/<i...## What does this MR do?
Adds settings `Shared` and `Path` to cache settings. Both will modify resulting URL as pointed out by @ayufan in #1897:
* By default, we keep current behavior, URLs have the form `/runner/<token>/project/<id>/<cache-key>`
* When `shared` flag is enabled, the runner segment is removed from the URL: `/project/<id>/<cache-key>`
* In any case, we could add a `Path` prefix to that URL: `<path>/runner/<token>/project/<id>/<cache-key>`
## Why was this MR needed?
Some users have problems when using autoscaling, because there's no way to remove runner token in the URL or add a prefix with the path. This change will enable both things preserving existing functionality.
## Are there points in the code the reviewer needs to double check?
`Path` allows spaces and other non-compatible URL characters that might cause problems or security issues. I've assumed we don't have to solve this here, given that `ServerAddress` for instance might introduce same problems - but it wouldn't harm to point that out in case we do that in the future. `cache_extractor.go` may be a good place to check for malformed URLs.
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Resolves #1897v1.9Kamil TrzcińśkiKamil Trzcińśkihttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/405Add uptime prometheus metric2016-12-21T21:44:31ZTomasz Maczukintomasz@gitlab.comAdd uptime prometheus metric## What does this MR do?
Adds another Prometheus metric.
## Why was this MR needed?
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/update...## What does this MR do?
Adds another Prometheus metric.
## Why was this MR needed?
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?v1.9Kamil TrzcińśkiKamil Trzcińśkihttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/397WIP added zsh completion function2017-01-19T10:03:11Zusername-removed-692238WIP added zsh completion functionThis adds a tab completion function for zsh.
The file is atm put at a more or less random location in the repository. After installation it should go to a location like `/usr/share/zsh/site-functions/_gitlab_ci_multi_runner`.
The sugge...This adds a tab completion function for zsh.
The file is atm put at a more or less random location in the repository. After installation it should go to a location like `/usr/share/zsh/site-functions/_gitlab_ci_multi_runner`.
The suggestions are mostly obtained by copy&pasting the `help` messages of `gitlab-ci-multi-runner`. In some obvious places (like `--kubernetes-ca-file`) the completion continues and suggests file names or (for `--working-directory`) directory names.
I did not (yet) make sure if it makes sense to suggest `--kubernetes*` options after `--docker*` options have been provided.
The function follows the example of the [yum completion in zsh](https://github.com/zsh-users/zsh/blob/master/Completion/Redhat/Command/_yum).
to be done:
* [ ] find a proper path to place the file and include in installation routine
* [ ] verify if it makes sense to provide different executor options in `register`or `run-single`
* [ ] the `help` messages mostly say `USAGE: command run-single [command options] [arguments...]` but only explain command options. Therefore no arguments are suggestedhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/389Kubernetes Namespace Overwrite2017-01-18T05:58:57Zusername-removed-764633Kubernetes Namespace Overwrite## What does this MR do?
This MR gives more flexibility for CI pipelines based on Kubernetes Runner. It adds a environment variable to overwrite the target Kubernetes namespace. Therefore, you can for instance, use `Helm`/`Landscaper`...## What does this MR do?
This MR gives more flexibility for CI pipelines based on Kubernetes Runner. It adds a environment variable to overwrite the target Kubernetes namespace. Therefore, you can for instance, use `Helm`/`Landscaper` to install Charts and create the set of services/pods that are required to execute the CI pipeline step, and also the ability to clean-up by deleting an namespace completely.
The namespace can only be overwritten if matches a configured regular expression (added to the runner configuration), to guarantee only designed namespace names can be employed. When this configuration is empty, this whole overwrite behavior is disabled.
## Why was this MR needed?
To create more flexible CI runs, to include `Helm` packages in a brand new namespace which will be deleted altogether afterwards. Using an overwrite mechanism avoid an extra steps to achieve the same goal.
## Does this MR meet the acceptance criteria?
- [x] Documentation created/updated;
- Tests
- [x] Added for this feature/bug;
- [x] All builds are passing;
- [x] Branch has no merge conflicts with `master`;
## What are the relevant issue numbers?
No issues related to this change.
v1.10Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/378Fix yaml-priv-reg link url2016-11-09T17:05:34Zusername-removed-819974Fix yaml-priv-reg link urlFix a link in the docs. The previous link had no reference to the linked content.Fix a link in the docs. The previous link had no reference to the linked content.https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/375WIP: Accept hostname/port combinations in image names2016-11-09T18:30:18ZStan HuWIP: Accept hostname/port combinations in image namesSpecifying a custom hostname and port causes name collisions to occur, preventing clones from occurring due to DNS failures. This happens if the hostname of the registry and the GitLab server are same but on different ports. For example:...Specifying a custom hostname and port causes name collisions to occur, preventing clones from occurring due to DNS failures. This happens if the hostname of the registry and the GitLab server are same but on different ports. For example:
1. `gitlab.example.com` is the GitLab server
2. `gitlab.example.com:4567` is the Registry
Previously an image named `gitlab.example.com:4567/namespace/project` would be aliased to `gitlab.example.com`, since everything after the colon would be stripped.
Closes #1434https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/360Benchmark the archive helpers2017-09-25T13:44:06ZNick ThomasBenchmark the archive helpers## What does this MR do?
It will help us speed up ZIP file extraction and creation, if we're lucky
## Why was this MR needed?
They're currently too slow, especially for large archives with many entries
## Are there points in ...## What does this MR do?
It will help us speed up ZIP file extraction and creation, if we're lucky
## Why was this MR needed?
They're currently too slow, especially for large archives with many entries
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Related to #1797https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/357[WIP] Add initial Prometheus metrics server to runner manager2016-10-26T13:12:04ZJulius Volz[WIP] Add initial Prometheus metrics server to runner manager## What does this MR do?
This adds a new `metrics_server` config option. If set to a listening address, a Prometheus HTTP server is started with the following metrics:
- runner business logic metrics (currently only number of currently...## What does this MR do?
This adds a new `metrics_server` config option. If set to a listening address, a Prometheus HTTP server is started with the following metrics:
- runner business logic metrics (currently only number of currently running builds)
- build version information
- Go process metrics (GC stats, goroutines, memstats, ...)
- general process metrics (FDs, memory, cpu usage, ...)
A full example `/metrics` output can be found here: https://gist.github.com/juliusv/5a1972875e9d80654c63d222c480f87e
This depends on the data race fix in https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/352, so it's based ontop of that.
The approach to integrating the `metrics_server` configuration option was taken from https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/219.
## Why was this MR needed?
We need better insight into the entire CI system, and currently we have no white-box metrics.
## Are there points in the code the reviewer needs to double check?
### General metrics approach
There were multiple different ways of approaching this. The more "traditional" Prometheus style is to just register all metrics from anywhere in the codebase in the global default Prometheus registry (similar to how logging is often treated as a global concern without dependency-injecting a logger; and analogous to using `http.Handle(...)` to register a handler with the global HTTP mux).
However, from reading the rest of this codebase, it seemed to me like an approach with explicit dependencies and less global state would be preferred by the authors. So I'm instantiating a custom Prometheus metrics registry and registering the Go and process metrics collectors with it manually (those are included automatically in the default metrics registry). I also am using the `prometheus.Collector` interface to encapsulate the metrics of a type, instead of the simpler approach of just defining global metrics. For an example of the latter, see the example at the top of https://godoc.org/github.com/prometheus/client_golang/prometheus. Generally, this all leads to somewhat more code, but more explicit dependencies and encapsulation.
### Configuration reloading
Like in https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/219, the metrics HTTP server does not allow for reconfiguration. In Go, it's also not trivial to stop an already running HTTP server. So right now, only whatever is configured at process startup takes effect. Is that ok? Should it be a command-line flag instead?
Please let me know if this approach is generally going in the right direction. I know there's many other metrics we will want to add over time, but maybe let's start out simple?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [x] All builds are passing
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
https://gitlab.com/gitlab-com/infrastructure/issues/543https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/350Graceful shutdown documentation2017-10-06T11:01:46Zusername-removed-778767Graceful shutdown documentation> This is a general Merge Request template. Consider to choose a template
> from the list above if it will match your case more.
## What does this MR do?
Improves the documentation surrounding graceful shutdown, resolving #1793
## Why...> This is a general Merge Request template. Consider to choose a template
> from the list above if it will match your case more.
## What does this MR do?
Improves the documentation surrounding graceful shutdown, resolving #1793
## Why was this MR needed?
When I attempted a graceful shutdown on Ubuntu 14.04 I thought it hadn't worked. It was confusing.
## Are there points in the code the reviewer needs to double check?
I don't personally use autoscaling, but I've updated the autoscaling documentation. There are some differences between my setup and what's documented - for example where the documentation says to use `killall -SIGQUIT gitlab-runner` I instead had to use `sudo killall -SIGQUIT gitlab-ci-multi-runner` - and I don't know if that's due to my setup or the documentation being outdated.
## Does this MR meet the acceptance criteria?
- [✔] Documentation created/updated
- Tests
- [✘] Added for this feature/bug
- [✔] All builds are passing
- [✔] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Resolves #1793v1.9Kamil TrzcińśkiKamil Trzcińśkihttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/342Building/Developing on OS X2017-04-22T01:16:32Zusername-removed-4901Building/Developing on OS X## What does this MR do?
Remove spaces on version identifier for builds on OS X.
## Why was this MR needed?
Currently, building using the Makefile on OS X is not possible due to spaces appearing in the flags passed to go.
The `wc` co...## What does this MR do?
Remove spaces on version identifier for builds on OS X.
## Why was this MR needed?
Currently, building using the Makefile on OS X is not possible due to spaces appearing in the flags passed to go.
The `wc` command on OS X adds whitespace to the beginning of the result of the ./ci/version script and causes the build to fail. This MR simply removes these spaces to allow development on that platform.
## Are there points in the code the reviewer needs to double check?
None - just one script file to check
## Does this MR meet the acceptance criteria?
- [x] Documentation created/updated
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [X] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Nonehttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/340WIP: X509 integration2017-04-25T12:28:25Zusername-removed-676625WIP: X509 integration## What does this MR do?
Adds the ability to configure and use x509 certificate and key pairs for TLS for connecting to gitlab servers which require authentication
## Why was this MR needed?
When making requests with gitlab-ci-multi-...## What does this MR do?
Adds the ability to configure and use x509 certificate and key pairs for TLS for connecting to gitlab servers which require authentication
## Why was this MR needed?
When making requests with gitlab-ci-multi-runner to gitlab servers which require certificate authentication it is presently impossible to do so. This allows for runner to utilize a certificate/key pair to make the connection.
## Are there points in the code the reviewer needs to double check?
I am not 100% clear on how this project handles documentation. I would ask the reviewer to take look and suggest how they would prefer documentation to be included. (ie which files to add too or if new files should be made etc...)
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers? none
## additional notes
Some tests seem to not work despite seemingly being unrelated to my code.
Specifically there are tests for the virtual box executor which pass the initial acceptance criteria but then fail because of a Minute time out fail
switching from minute to hour gets these tests to pass but I am not sure why they are failing in the first place.
the docker executor test seems to time out. This started to happen after a rebase to 1-6 stable removed a dependencyhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/338WIP: Kubernetes pre-build container2016-12-22T11:04:54Zusername-removed-357684WIP: Kubernetes pre-build container## What does this MR do?
It uses the predefined images for Pre-/Post-Build steps (like docker executor does) as sidecar container for the runner Pods.
I need some help in how to handle the predefined images. In docker executor has them ...## What does this MR do?
It uses the predefined images for Pre-/Post-Build steps (like docker executor does) as sidecar container for the runner Pods.
I need some help in how to handle the predefined images. In docker executor has them embedded but I don't think this is possible for kubernetes.
Is there any reason to not push the predefined images to Docker Hub and pull from there? This is what I did for testing:
```bash
for arch in arm x86_64; do
wget https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/master/docker/prebuilt-$arch.tar.xz -O /tmp/prebuilt-$arch.tar.xz
docker import prebuilt-$arch.tar.xz jaymedh/gitlab-ci-runner-prebuild-$arch:latest
docker push jaymedh/gitlab-ci-runner-prebuild-$arch:latest
done
```
## Why was this MR needed?
It removes the need to have `git` and Gitlab-Runner binary in the user images. Thus makes it easier to run builds on user provided images.
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?