gitlab-runner merge requestshttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests2016-09-19T06:11:59Zhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/130Remove user-mode warning when runned in system-mode2016-09-19T06:11:59ZTomasz Maczukintomasz@gitlab.comRemove user-mode warning when runned in system-modeThis MR removes warning: `The user-mode requires you to manually start builds processing (...)` when a command is executed in system-mode.
Closes #1179This MR removes warning: `The user-mode requires you to manually start builds processing (...)` when a command is executed in system-mode.
Closes #1179https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/131Upload packages to packagecloud when Bleeding Edge build was done on master b...2016-09-19T06:11:59ZTomasz Maczukintomasz@gitlab.comUpload packages to packagecloud when Bleeding Edge build was done on master branchTomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/128avoid duplication2016-09-19T06:11:59Zusername-removed-163557avoid duplicationuse existing outputuse existing outputhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/114Create user as a system account.2016-09-19T06:11:59Zusername-removed-63055Create user as a system account.This is an account for a service rather than a human being and should therefore be created as a system user. This puts it in the system UID and GID range.This is an account for a service rather than a human being and should therefore be created as a system user. This puts it in the system UID and GID range.v1.2https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/127Use different detection mechanism2016-09-19T06:11:59ZKamil TrzcińśkiUse different detection mechanismv1.1https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/121Update the OSX service instruction2016-09-19T06:11:58ZKamil TrzcińśkiUpdate the OSX service instructionThis tries to answer and notice most common problems when using Runner as an service on OSX.
We should add to a blog post information about this findings and advise any OSX user to upgrade startup script:
```
gitlab-runner uninstall...This tries to answer and notice most common problems when using Runner as an service on OSX.
We should add to a blog post information about this findings and advise any OSX user to upgrade startup script:
```
gitlab-runner uninstall
gitlab-runner install
gitlab-runner start
```
The binary:
* https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/osx-launch-agents/binaries/gitlab-ci-multi-runner-darwin-amd64
@axil Could you review?
Fixes https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/972 https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/217 https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1143 https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/122v1.1Kamil TrzcińśkiKamil Trzcińśkihttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/118Add OpenBSD build support2016-09-19T06:11:58Zusername-removed-438536Add OpenBSD build supportThis commit makes the complete binary compile on OpenBSD.This commit makes the complete binary compile on OpenBSD.https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/115Added docker pull policy2016-09-19T06:11:57ZKamil TrzcińśkiAdded docker pull policyThis removes the `image_ttl` from `[runners.docker]` and replaces it with `pull_policy`.
The values:
* `always` - pull image every time **default**,
* `if-not-present` - pull image for the first time only,
* `never` - don't pull im...This removes the `image_ttl` from `[runners.docker]` and replaces it with `pull_policy`.
The values:
* `always` - pull image every time **default**,
* `if-not-present` - pull image for the first time only,
* `never` - don't pull image, fail if the image is not present,
@tmaczukin Please reviewv1.1Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/93Fixes #1074 Generate random IP addresses for port forwarding2016-09-19T06:11:57Zusername-removed-405155Fixes #1074 Generate random IP addresses for port forwardingThis merge request lowers the chance of a race condition with finding free port numbers for the virtualbox port fowarding. It uses the fact that binding to port 0 forces the kernel to allocate a random high port number, and the kernel pr...This merge request lowers the chance of a race condition with finding free port numbers for the virtualbox port fowarding. It uses the fact that binding to port 0 forces the kernel to allocate a random high port number, and the kernel prefers ones that have not been recently used.
This random port is not bound by any other process at the time, or we wouldn't have received it. Do check it against the used ports as listed by `vboxmanage list vms -l` in case some virtual machines are not currently running.v1.1Kamil TrzcińśkiKamil Trzcińśkihttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/108Add missing Git in alpine version2016-09-19T06:11:56Zusername-removed-82951Add missing Git in alpine versionWith the ```gitlab-runner:alpine``` version, this error is thrown when a build is runs.
```
gitlab-ci-multi-runner 1.0.4 (014aa8c)
Using Shell executor...
Running on 084238b313b4...
Cloning repository...
bash: line 34: git: comma...With the ```gitlab-runner:alpine``` version, this error is thrown when a build is runs.
```
gitlab-ci-multi-runner 1.0.4 (014aa8c)
Using Shell executor...
Running on 084238b313b4...
Cloning repository...
bash: line 34: git: command not found
ERROR: Build failed with: exit status 1
```https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/109report process start failure2016-09-19T06:11:56Zusername-removed-378265report process start failureWhen the shell executor fails to start a process, it currently eats the error, reporting only a static string. This change exposes the underlying problem.When the shell executor fails to start a process, it currently eats the error, reporting only a static string. This change exposes the underlying problem.https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/65add network_mode option to docker runner2016-09-19T06:11:56Zusername-removed-364336add network_mode option to docker runnerThis addition allows specifiying a network_mode in config.toml so that build containers can be run in specific docker networks.This addition allows specifiying a network_mode in config.toml so that build containers can be run in specific docker networks.v1.1Kamil TrzcińśkiKamil Trzcińśkihttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/53Runner Autoscale through docker-machine2016-09-19T06:11:55ZKamil TrzcińśkiRunner Autoscale through docker-machine## What is Docker Machine?
> Machine lets you create Docker hosts on your computer, on cloud providers, and inside your own data center. It automatically creates hosts, installs Docker on them, then configures the docker client to tal...## What is Docker Machine?
> Machine lets you create Docker hosts on your computer, on cloud providers, and inside your own data center. It automatically creates hosts, installs Docker on them, then configures the docker client to talk to them. A “machine” is the combination of a Docker host and a configured client.
Source: https://docs.docker.com/machine/
## Why Docker Machine?
Is easy to use. Is well documented. Is well supported and constantly extended. It supports almost any cloud provider or virtualization infrastructure. We need minimal amount of changes to support Docker Machine: machine enumeration and inspection. We don't need to implement any "cloud specific" features.
## GitLab Runner and Docker Machine sitting on tree
This MR adds support for Docker Machine in GitLab Runner allowing to automatically spin-up Docker-enabled hosts on any provider supported by Docker Machine: DigitalOcean, VirtualBox, Azure...
This allows to have automatically scalable cluster of runners managed by one GitLab Runner instance. You can set to have a maximum of 100 jobs running in parallel. The Docker Machine integration will create Cloud Instances on demand to accommodate the load. After defined period (it can be 1h) the machines will be destroyed and a new one will be created when needed.
We use one GitLab Runner to manage all jobs.
## How it works?
1. The GitLab Runner requests the Job from coordinator
2. The GitLab Runner checks available docker machines and takes the first one free
3. If no machine is found a new one is created
4. The Job is run on provisioned machine
5. After the Job is run the disk space is checked and if it goes below certain level the machine is removed
6. Otherwise the machine is added to list of machines that can be reused for other jobs
## How to enable Docker Machine configuration
The simplest `config.toml`:
```
concurrent = 50
[[runners]]
url = "https://gitlab.com/ci"
token = "XYZ"
name = "YZX"
executor = "docker+machine"
limit = 10
[runners.docker]
image = "ruby:2.1"
[runners.machine]
IdleCount = 5
IdleTime = 600
MaxBuilds = 100
MachineDriver = "digitalocean"
MachineName = "auto-scale-%s"
MachineOptions = [
"digitalocean-image=coreos-beta",
"digitalocean-ssh-user=core",
"digitalocean-access-token=DO_ACCESS_TOKEN",,
"digitalocean-region=nyc2",
"digitalocean-size=4gb",
"digitalocean-private-networking",
"engine-registry-mirror=http://10.128.11.79:34723"
]
```
The above configuration will run up to 10 jobs and it will remove machines after 600 seconds of inactivity. It leave at most 5 nodes waiting to process the jobs. The machines are created on DigitalOcean, using CoreOS image. The boot time of CoreOS VM is between 40-60s.
Other MachineOptions:
```
docker-machine create -d digitalocean --help
```
## Limitations
The runner cache is tied to one node. It can be solved by exposing cache as internal runner HTTP service.
**No longer true: if this is enabled: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/88**
## Considerations
The provisioned machines should not hold any data. The machines should be disposable if not needed. Having to provision a new node should not make a difference for a running a next stage build.
v1.1Kamil TrzcińśkiKamil Trzcińśkihttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/67Add post `apt-get install` cleanup2016-09-19T06:11:50Zusername-removed-222409Add post `apt-get install` cleanupImplements best practices from
https://docs.docker.com/engine/articles/dockerfile_best-practices/#apt-get
Fixes #974Implements best practices from
https://docs.docker.com/engine/articles/dockerfile_best-practices/#apt-get
Fixes #974https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/311Don't consult or change the global and system git configuration2016-09-14T16:45:45ZNick ThomasDon't consult or change the global and system git configuration## What does this MR do?
Writes a hard-coded `gitconfig` file per build and sets it in `$GIT_CONFIG`.
## Why was this MR needed?
Previously, we were running `git config --global fetch.recurseSubmodules false" to disable recursive subm...## What does this MR do?
Writes a hard-coded `gitconfig` file per build and sets it in `$GIT_CONFIG`.
## Why was this MR needed?
Previously, we were running `git config --global fetch.recurseSubmodules false" to disable recursive submodule fetches. This has a few problems:
* It modifies global state for the user the runner is under, which is unexpected in some environments
* Concurrent builds sometimes fail as `git config` doesn't cleanly handle concurrent writes to the same file.
This change switches to a separate gitconfig file per-build, with hardcoded values. This solves both issues, at the cost of removing support for setting custom options for the runner in /etc/gitconfig or ~/.gitconfig.
## Are there points in the code the reviewer needs to double check?
* Do we need to support /etc/gitconfig or ~/.gitconfig (or the Windows equivalents)? Do we know if users are relying on this secret feature at the moment?
## Does this MR meet the acceptance criteria?
- [ ] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/CHANGELOG.md) entry added
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
Closes #1469Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/307Remove yaml v12016-09-14T11:50:45ZNick ThomasRemove yaml v1## What does this MR do?
Replaces`gopkg.in/yaml.v1` with `gopkg.in/yaml.v2`
## Why was this MR needed?
We already vendor in gopkg.in/yaml.v2 and downstreams tend not to have v1 packaged any more.
## Are there points in the code the r...## What does this MR do?
Replaces`gopkg.in/yaml.v1` with `gopkg.in/yaml.v2`
## Why was this MR needed?
We already vendor in gopkg.in/yaml.v2 and downstreams tend not to have v1 packaged any more.
## Are there points in the code the reviewer needs to double check?
## Does this MR meet the acceptance criteria?
- [ ] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/CHANGELOG.md) entry added
- [ ] Documentation created/updated
- Tests
- [ ] Added for this feature/bug
- [ ] All builds are passing
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)
## What are the relevant issue numbers?
* Closes #1499
* Replaces !247v1.6Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/175Update Go instructions in README2016-09-14T11:50:44Zusername-removed-380317Update Go instructions in READMEGo 1.4.x no longer builds this project; Update to install 1.5.x.Go 1.4.x no longer builds this project; Update to install 1.5.x.https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/291Fixed some spelling/grammar mistakes.2016-09-14T11:50:44Zusername-removed-625735Fixed some spelling/grammar mistakes.I found a simple typo, and then started to proofread the whole FAQ in order to improve the text.
This change is purely cosmetic in nature.I found a simple typo, and then started to proofread the whole FAQ in order to improve the text.
This change is purely cosmetic in nature.https://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/295Generate a BuildError when Docker/Kubernetes image is missing2016-09-14T11:50:43ZKamil TrzcińśkiGenerate a BuildError when Docker/Kubernetes image is missingFixes https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1629Fixes https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1629v1.6Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.comhttps://staging.gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/193Support shared builds volume (docker executor)2016-09-09T12:50:44Zusername-removed-158815Support shared builds volume (docker executor)Reasons for shared /builds volume are outlined in https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/66#note_3222227.
This MR does not introduce any configuration options, instead it checks whether /builds is mounted...Reasons for shared /builds volume are outlined in https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/merge_requests/66#note_3222227.
This MR does not introduce any configuration options, instead it checks whether /builds is mounted as a host volume and if it is - sets shared_builds_dir to true & prevents /builds/{group} volume from being created.
Usage:
```sh
docker exec -it gitlab-runner gitlab-runner register --non-interactive --description <description> --url http://gitlab.example.com/ci --registration-token <registration-token> --executor docker --tag-list docker --docker-image <docker-image> --docker-volumes /var/run/docker.sock:/var/run/docker.sock --docker-volumes /builds:/builds
```
Let me know if you want anything to be changed. Thank you.v1.5Tomasz Maczukintomasz@gitlab.comTomasz Maczukintomasz@gitlab.com