"fatal: Not a git repository" with concurrent jobs - might be a race condition, Virtualbox executor
I have a project which had one runner, to build on Ubuntu 14.04 using a VirtualBox executor. That worked without error. I also need to build on Ubuntu 12.04, so I set up a new VirtualBox image, and registered a new runner. gitlab-ci-multi-runner verify
reports they're both fine, and it can take jobs. The 14.04 image has the tag ubuntu1404
and the 12.04 image has the tag ubuntu1204
.
When I make a change, two builds are triggered, as I'd hoped. Two virtual machines are created - going by the names provided by vboxmanage list runningvms
, one is the Ubuntu 12.04 VM, and the other is the Ubuntu 14.04 VM.
However, both builds fail, because they both appear to be running on the same VM.
If my runner is a server called runner01
, and the two VMs are builder-ubuntu-1204
and builder-ubuntu-1404
, then the 12.04 job says:
gitlab-ci-multi-runner 1.0.2 (ea19241)
Using VirtualBox version 4.3.34r104062 executor...
Creating new VM...
Waiting VM to become responsive...
Starting SSH command...
Running on builder-ubuntu-1204 via runner01...
Fetching changes...
fatal: Not a git repository (or any of the parent directories): .git
ERROR: Build failed with: Process exited with: 1. Reason was: ()
And the 14.04 job says:
gitlab-ci-multi-runner 1.0.2 (ea19241)
Using VirtualBox version 4.3.34r104062 executor...
Creating new VM...
Waiting VM to become responsive...
Starting SSH command...
Running on builder-ubuntu-1204 via runner01...
Cloning repository...
Cloning into 'builds/group1/project1'...
ERROR: Build failed with: wait: remote command exited without exit status or exit signal
Even though the 14.04 and 12.04 VMs are created, both jobs claim they're running on builder-ubuntu-1204, which explains the git errors.
If I manually restart both builds, they start on the VM they're meant to, and both end up passing. I suspect that this is because the time between me clicking 'Retry' on one, changing the tab and clicking 'Retry' again is long enough that the first can sort out its metadata, and that when the jobs are triggered automatically, the right VMs are started but there is confusion about where to send the job commands.
I have a workaround, where I create multiple build stages, which prevents them running concurrently. It would be good to get concurrent builds working, though.