Establish mechanism for communication between CE/EE and git-access-daemon

One of the things that is not clear to me yet is how GitLab will interact with this project. Some options are:

Make the client a git binary substitute

Pros:

• Completely transparent for our current workflow, minimum adjustment needed.
• Makes sure you don't miss any points where git is accessed (for example if you were to just look for Gitlab::Git classes you'd miss things like popen('git ...'))

Cons:

• Obfuscated: Not very clear that you're actually going now through an API instead of to the filesystem. This is the other side of the "Pro" of a transparent workflow: you should probably approach an API call different than a FS access on your code. Although, as @pcarranza noted somewhere else, you're actually going through a network anyways because our FS is distributed.
• Error handling becomes a bit more awkward, having to read status from the command output

Refactor app/models/repository.rb to be the client

Pros:

• Clear and cleaner architecture: GitLab -> git-access-daemon, no middleman, so the CE code can handle neatly all caveats and errors.

Cons:

• You have to be extra careful to refactor everything (double check EE, if we go down this route we'll miss something there), since an unhandled git access could lead to inconsistencies.
• We introduce some complexity to CE (an API client), which goes against what we're trying to do.

Make gitlab_git the client

Pros:

• You'd refactor gitlab_git, but GitLab CE/EE would require no big changes except for removing all caching logic

Cons:

• Subjective, but this options seems awkward to me. It seems like relying on a component our solution should be getting rid of. But it might actually make sense if you stop seeing gitlab_git as a library and start seeing it as an adapter.

/cc @pcarranza @yorickpeterse @ahmadsherif

@eReGeBe

Regarding Make the client a git binary substitute why would you need to handle status differently? if it's transparent it should also return the status code that the original command returned.

Regarding the other options, I really don't know.

I think that it's important to note one thing: on the one hand I expect to move those processes that we will still be getting -> git commands through ssh, they will never go away and we will need to serve them. Here what we can do is call this client as if it is git, and take the load on the deamon on the other side. But it still has to look like regular git.

On the other hand, the API that we could call from the application, as I mentioned before, it has nothing to do with the git command itself and we should treat that as a separate thing. How we do it? well that's a different discussion (I'm ok with making it restful, BTW)

So I think we need 2 different and well defined interactions, one over a simple streaming network socket (the first one) and another that is much more command based (REST?)

The reason why I think a git binary substitute would hide problems is because we are not going to support all git commands (since there are so many), so we'll have to choose if the command received is going to go to the git-access-daemon server or to the native git binary. This would not make it clear when are we and when are we not taking advantage of the caching mechanism in the daemon, so something could slip through (maybe because someone adds a feature that uses X git command, which we don't support yet in the server so we pass it to git native, and through git's intricacies it happens to leave the cache in an inconsistent state).

Subjectively, I just fear there's something else we're going to miss when trying to act like git. I think having a separate well defined client would save us from having two ways to communicate with the daemon and allow to have just one way with a specific interface that enables the developer to be aware of how to take the most advantage out of the daemon.

I think we are getting lost in translation here. @eReGeBe let me know when you are around and we can discuss this over a call. Else we are going to be writing long texts explaining each other and we are probably saying and thinking similar things.

I spoke with @pcarranza and clarified that when talking about two interactions we're talking about:

• Communication between shell/workhorse and the daemon: This is where we're talking about a "git-binary-substitute", by which we mean not someone pretending to be git but a new binary (maybe git-daemon-client) which will be the one to interact with in shell/workhorse. This is the first thing we want to implement

• Communication between CE/EE and the daemon: This is where we'll probably be using a REST API.

changed milestone to %Simple Prototype

I am not sure if gitaly-client is a good idea. You'd have to install this and make sure that gitlab-shell and gitlab-workhorse can find the executable.

For gitlab-workhorse I would suggest HTTP requests. Take Git HTTP as an example.

• HTTP request reaches workhorse
• workhorse asks gitlab-rails if this is allowed
• rails says: go ahead, talk to gitaly
• workhorse proxies to gitaly

This makes more sense to me than workhorse starting a gitaly-client process and piping date in/out of that process. Go has a perfectly fine HTTP client we can use for this.

For gitlab-shell there are two connections I can think of: hooks and SSH sessions. We can use HTTP requests and websockets for those, respectively.

Am I missing something, or does that cover it?

Also see #16 (closed) (access control discussion).

mentioned in issue #7 (closed)

@jacobvosmaer-gitlab the workhorse side makes sense.

The one I'm missing is when we get an ssh connection in, my understanding is that we should do something like what workhorse does:

• SSH connection reaches a worker
• openssh-server uses gitlab-shell to prompt for authorized keys
• openssh-server uses gitlab-shell again to prompt for /allowed
• openssh replaces the executable that ssh is trying to run for git gitaly-client which forwards the git execution to the gitaly-server.

What I mean is that we need to remove the git execution from the worker (even though we don't do it immediately) to remove the NFS mounts from the workers.

This could happen that we are using the same http endpoint at workhorse. but it looked like less of a code disruption to just leverage the ability to run a process to start with something simpler.

It could also be that I'm missing something else.

Closing this one as we changed the direction about how are we going to communicate.

Continue in https://gitlab.com/gitlab-org/gitaly/issues/21

closed

To answer @pcarranza's comment above:

• openssh-server uses gitlab-shell again to prompt for /allowed
• openssh replaces the executable that ssh is trying to run for git gitaly-client which forwards the git execution to the gitaly-server.

This is misunderstanding how gitlab-shell works now. What happens is:

• openssh-server records the original command from the client in an env variable and launches gitlab-shell/bin/gitlab-shell key-1234
• the gitlab-shell process queries the /internal/allowed API
• if OK, gitlab-shell exec's into e.g. git upload-pack

My proposal was to replace the last step with: gitlab-shell creates a websocket connection to gitaly (http://something/ssh/upload-pack) and copies the streams of the SSH session back and forth, acting as a proxy.