Retry failed repository clone/fetch

As seen on https://gitlab.com/gitlab-org/gitlab-ce/builds/6557396:

Fetching changes for fix/cycle-analytics-plan-issue with git depth set to 20...
[SNIP]
HEAD is now at ecbff8d Rephrase some system notes to be compatible with new system note style
error: RPC failed; HTTP 502 curl 22 The requested URL returned error: 502 Bad Gateway
fatal: The remote end hung up unexpectedly
ERROR: Build failed: exit code 1

And https://gitlab.com/gitlab-org/gitlab-ce/builds/6563389:

Cloning repository for master with git depth set to 20...
Cloning into '/builds/gitlab-org/gitlab-ce'...
error: RPC failed; HTTP 502 curl 22 The requested URL returned error: 502 Bad Gateway
fatal: The remote end hung up unexpectedly
ERROR: Build failed: exit code 1

/cc @ayufan @tmaczukin

Milestone changed to %v1.9

Added improvement label

Added ~1009246 label

I think this is due to https://gitlab.com/gitlab-com/infrastructure/issues/685?

@stanhu I think we still decided to retry in case of any network errors/slowness...

@ayufan what has to be done for this to be implemented? I wonder if I could help even if I'm not too familiar with CI. Is this also touching the TestEnv calls to git clone?

@jameslopez

We need to retry a pre-build step in GitLab Runner.

If you can work on this we can guide you how to do that.

This is the line that is responsible for clonning and downloading artifacts: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/common/build.go#L167

@ayufan thanks a lot! I'll have a look...

Mentioned in merge request !399 (merged)

Reassigned to @jameslopez

Added Doing label

closed via merge request !399 (merged)