Handle GL_REPOSITORY env variable and use it in api calls

Gitlab-Shell implementation of https://gitlab.com/gitlab-org/gitlab-ce/issues/29925

/cc @jacobvosmaer-gitlab

added 1 commit

• d55427de - Handle GL_PROJECT env variable and use it in api calls

Compare with previous version

added 2 commits

• 3db8157c - Use gl-project if present to enqueue PostReceive sidekiq jobs
• 86cd5158 - Update changelog for GL_PROJECT handling

Compare with previous version

unmarked as a Work In Progress

assigned to @jacobvosmaer-gitlab

Looks good.

assigned to @eReGeBe

changed title from Handle GL_PROJECT env variable and use it in api calls to Handle GL_REPOSITORY env variable and use it in api calls

resolved all discussions

assigned to @jacobvosmaer-gitlab

@eReGeBe I think we still need to replace GL_PROJECT in quite a few places.

assigned to @eReGeBe

Sorry, I missed the rename. Done now.

assigned to @jacobvosmaer-gitlab

@eReGeBe I think you need to fix up that search-and-replace error. Otherwise, this can go to a maintaner.

assigned to @eReGeBe

added 5 commits

• b378fff8...1cf14770 - 2 commits from branch master
• fec60cf5 - Handle GL_REPOSITORY env variable and use it in api calls
• 26a3e4bc - Setup use of gl-repository to enqueue PostReceive sidekiq jobs
• 826d5e75 - Update changelog for GL_REPOSITORY handling

Compare with previous version

assigned to @smcgivern

added Gitaly label

mentioned in issue gitaly#201 (closed)

@eReGeBe thanks, just a couple of questions!

assigned to @eReGeBe

@eReGeBe before this goes in: @andrewn had a good idea in a discussion earlier.

Instead of changing the signature of the PostReceive worker, we could create a new worker PostReceiveV2. Then here in gitlab-shell we would enqueue messages on the queue of PostReceiveV2, and we would not have to worry about the change-over as much. If we do it this way we would need fewer intermediate releases. What do you think?

mentioned in issue gitaly#218

@jacobvosmaer-gitlab what would the plan be in that case?:

• 9.3: Dual enqueue PostReceive and PostReceive v2
• 9.4: Only enqueue PostReceive v2

?

Could this be done with our current plan in patch releases?:

• 9.2.1: Update shell to this version
• 9.2.2: Update shell to @gl_repository || @repo_path
• 9.2.3: Update shell to @gl_repository

Or is that something that can't be done in a patch release @smcgivern?

That's up to @timothyandrew and @selfup!

@jacobvosmaer-gitlab doesn't that mean keeping PostReceive and PostReceiveV2 around together for a while, and then moving PostReceiveV2 back to PostReceive once we're 'done'? I'm not sure if that is simpler, but maybe I've misunderstood.

@eReGeBe: we follow this pattern a lot in Gitter (where we use resque, similar to Sidekiq):

• 9.2: Listen to PostReceive. Enqueue to PostReceive
• 9.3: Listen to PostReceive and PostReceiveV2 (separate workers, expecting differently shaped messages). 9.3 workers enqueue to PostReceiveV2. During the deployment process, some 9.2 workers will continue to enqueue to PostReceive, and 9.3 workers will process those messages, but these queues should empty within minutes or hours of the deploy taking place.
• Post 9.3 release cut: PostReceive listener removed from the codebase
• 9.4: Only listen to PostReceiveV2. Enqueue to PostReceiveV2

@andrewn right, but don't forget that customers don't have to deploy 9.2 or 9.3 at all if they don't want to. Also, it's going to be weird if we just have a PostReceiveV2 - my assumption was that we'd want to rename that back to the old name at some point.

Basically, I don't see how this needs fewer intermediate steps

@andrewn right, I get the methodology. But there's also the fact that https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10992 got into 9.2 with the PostReceive worker handling both param types. I think we should follow our current plan this time, and if @timothyandrew and @selfup agree with changing GITLAB_SHELL_VERSION on patch releases we can complete the cycle in 9.3. As @smcgivern pointed out, users are free to skip any minor or major release, so we're only really taking care that the transition is smooth on gitlab.com, where we deploy every version.

added 3 commits

• ee259653 - Handle GL_REPOSITORY env variable and use it in api calls
• b3c8ffe6 - Setup use of gl-repository to enqueue PostReceive sidekiq jobs
• cba67a74 - Update changelog for GL_REPOSITORY handling

Compare with previous version

resolved all discussions

I agreed with @andrewn that it's better we continue this down the path we already started, and consider his proposal (which I like) for the next time a change of this sort is required. On that note, code updated @smcgivern !

assigned to @smcgivern

resolved all discussions

merged

mentioned in commit c40273b0

Thanks @eReGeBe! Do you want to take https://gitlab.com/gitlab-org/gitlab-shell#releasing-a-new-version?

@smcgivern I don't have master access to this project :(

@smcgivern would you mind handling https://gitlab.com/gitlab-org/gitlab-shell#releasing-a-new-version ? (I actually thought I was master on this project but ¯\(ツ)/¯

@jacobvosmaer-gitlab it looks like Sean is out this week. Can you release a new version of shell to continue with the migration plan?

@eReGeBe do you still need this to be done? I see we have a 5.0.4 tag now.

@smcgivern yes, the new tag was released thanks

@eReGeBe is there an issue for the follow-up steps needed for this signature change?

mentioned in issue gitaly#245 (closed)

@jacobvosmaer-gitlab https://gitlab.com/gitlab-org/gitlab-shell/issues/83

mentioned in issue gitaly#349