Use gitaly-upload-pack and gitaly-receive-pack

Closes https://gitlab.com/gitlab-org/gitaly/issues/193

We want to use Go Gitaly client executables in gitlab-shell because we cannot reliably use Ruby gems with C native extensions in Ruby executables launched by SSHD. This change adds such Go executables, for now without Gitaly support but that should change soon.

This change makes bin/compile a mandatory installation step. All git pull and git push sessions over SSH will start flowing through these Go executables.

Needs more work on logging, more structure for config parsing, and gitaly-receive-pack.

assigned to @jacobvosmaer-gitlab

I'm happy with structure and functionality now. Still needs a way to run go tests in CI.

The Go tests are working now. Still missing: some sort of Ruby test coverage for the hand-off to Go.

added 1 commit

• c138ee12 - Use gitaly-upload-pack and gitaly-receive-pack

Compare with previous version

changed title from WIP gitaly ssh command wrapper to WIP Use gitaly-upload-pack and gitaly-receive-pack

Realized this is still work-in-progress because everything breaks after this is merged on systems that don't compile the Go executables. We first need to update gitlab-ce and omnibus to run bin/compile.

mentioned in issue omnibus-gitlab#2304 (closed)

added 3 commits

• 1272d992 - Change signature of logger.Fatal
• b57da39b - Check number of arguments
• 67bc4f5c - Check go formatting in CI

Compare with previous version

resolved all discussions

added 1 commit

• 8964da0b - Explain chdir

Compare with previous version

This is WIP because gitlab-ce!11039 and omnibus-gitlab#2304 (closed) need to be resolved before it is safe to merge this. Otherwise it is ready for review.

assigned to @DouweM

Note to reviewers: there are relevant diffs below the go/vendor bits.

@jacobvosmaer-gitlab Nice work, looks fairly straightforward! One suggestion.

assigned to @jacobvosmaer-gitlab

added 1 commit

• de018a56 - Make gitaly executable mapping clearer

Compare with previous version

assigned to @DouweM

resolved all discussions

@jacobvosmaer-gitlab LGTM! Let me know when those other issues are resolved and this can be merged.

assigned to @jacobvosmaer-gitlab

Thanks @DouweM will assign back to you when ready.

Locked in WIP by https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2304

Talking with @andrewn I realized it might be better to make the use of gitaly-{upload,receive}-pack conditional on the API response received from gitlab-ce. That way we can merge this independently of omnibus-gitlab#2304 (closed). And we want an on/off mechanism anyway. I was originally planning to have on/off inside gitaly-upload-pack but if we put it in bin/gitlab-shell it makes our life a little easier.

This means I will update this MR and re-submit it for review.

added 13 commits

• de018a56...285c061e - 7 commits from branch master
• 25a32cfa - Use gitaly-upload-pack and gitaly-receive-pack
• 33d583bf - Change signature of logger.Fatal
• e3dc9c76 - Check number of arguments
• 5fd923bd - Check go formatting in CI
• 2a163a17 - Explain chdir
• f3b83553 - Make gitaly executable mapping clearer

Compare with previous version

unmarked as a Work In Progress

I have not had the time yet to update this MR and now omnibus has the bin/compile step. That means this can be merged without breaking anything (knock on wood :) ).

assigned to @DouweM

merged

mentioned in commit db96f724

@jacobvosmaer-gitlab Nice work :)

mentioned in issue gitaly#124 (closed)

mentioned in issue gitaly#349