Add ability to configure a global set of custom hooks

Description including problem, use cases, benefits, and/or goals

Organisations would like to apply custom git hooks to all projects, present and future. This makes it easier to enforce a very specific workflow.

Right now, this requires manual editing the server for each new project.

Proposal

We can do global custom_hooks directory in gitlab-shell. There you can put custom hooks that will be symlinked to every new project created. If you don't want a hook in certain project - just manually remove symlink. Its also up to author to make a proper error messages in hooks to make experience more friendly.

A rake task that would show which projects have or lack the symlink would make this significantly easier.

original issue

Some organizations configure custom hooks that they wish to apply to all projects - present and future. This is especially useful for pre-receive hooks - adding some policy check before accepting a push. This could potentially be accomplished through a symlink or we could add an additional call to global hooks before project-specific custom hooks.

This feature was requested by a large prospective organization.

cc/ @dzaporozhets What do you think?

@dblessing You're positive this organization will have these hooks on every repo, no exceptions?

@sytses That's what they've told us. They have a series of pre-checks they run on every project to enforce policy. It's a bit of a specific use-case, but I can see how it might be useful to some other organizations. I also don't think it would be too much effort to do it.

@JobV It would be great to get your feedback on this feature request as well. This request is from an India based company that provides an online marketplace. They have around 2000 developers and this is a critical feature request for their current workflow. (This could be a good Enterprise upgrade type feature).

@ZachLong @dblessing for extensive checks, they're probably better of using CI and a good merge flow.

None the less, I can imagine an organisation wanting this, although it's not a very good idea.

@dzaporozhets your thoughts on the feasibility of this?

@dblessing I have done this use my function

use my patch and custom_hooks.git project in gitlab it can hold groups hooks and every project ～

see #34

cc/ @sytses @ZachLong @JobV @dzaporozhets

by the way how can I submit my merge request~ @dzaporozhets @dblessing https://gitlab.com/gdbdzgd/gitlab-shell/commit/4afb101a85bdda67e28e707b66b140a457e8fc55

we can add an option in web control global custom_hooks [enable/disable]

https://gitlab.com/gdbdzgd/gitlab-shell/commit/4afb101a85bdda67e28e707b66b140a457e8fc55

Hi @gdbdzgd

For something like global custom hooks we need to implement it in the same way as the current single project custom hooks. This means filesystem configuration only, no application settings. This is a low-level feature.

If you still wish to send a merge request you can do so by forking the project and then open a merge request against the main repository master branch.

Implementation suggestion from a comment on IRC: "What about having a directory of global custom hooks and linking them (or a specified subset of them) into newly-created repos by default? Allows global custom hooks as a starting point but repos could customize which ones they need after. Would also be easy to see what hooks will be run (anything in the project's custom_hooks dir - normal script or link to a global hook)"

Pasting here for later. I haven't had time to digest it.

@JobV Will you please schedule?

@dblessing I'm not excited about the proposals as they will stifle moving quickly with new projects.

I also have to hear from @dzaporozhets whether this is easily possible.

@JobV What do you mean 'stifle moving quickly with new projects'?

From a technical perspective it should be relatively easy. We have the logic for custom hooks, now it just needs to execute by looking in one additional location.

@dblessing @JobV are we talking about feature like this http://doc.gitlab.com/ce/hooks/custom_hooks.html instead of per-project customer wants certain hook be symlink to all projects from gitlab-shell directory?

@dzaporozhets yes.

@dblessing because custom hooks set restrictions to new projects without being visible immediately to the person starting the project.

Say I make a custom hook that disallows you to commit files ending with .test. As a new employee I might make a little test project and try to push something I call test.test. When I push it, my push gets rejected without any mention of the reason.

Anyway, I don't really like pre-commit hooks. I'll let @dzaporozhets give his opinion, because I do understand the usecase of the customer.

@JobV Thanks. I understand where you're coming from. For the most part, I'm also against putting too much logic in custom hooks like this. It's surprising the number of customers who have complex hooks that do things like check JIRA. They're not just checking that an issue is mentioned, but that the issue is in the correct state before pushing. It seems like something so specific and yet I've heard it from multiple customers.

@dblessing @JobV in general I am not big fan of this idea. I would rather implement more of this http://doc.gitlab.com/ee/git_hooks/git_hooks.html. However I understand it might now work for all customers.

We can do global custom_hooks directory in gitlab-shell. There you can put custom hooks that will be symlinked to every new project created. If you don't want a hook in certain project - just manually remove symlink. Its also up to author to make a proper error messages in hooks to make experience more friendly.

Basically pure gitlab-shell feature.

Thanks @dzaporozhets. I'll add your proposal to the description.

@dblessing can you see if this works for the customer?

@JobV Yes, this will work. It may be useful to have rake tasks to show which projects don't have the global link, and add it to all or some.

Milestone changed to 8.11

Added ~243323 feature proposal labels

Requested: https://gitlab.zendesk.com/agent/tickets/23584

My company would also be interested in having a set of global custom hooks, mainly for mailing notifications to a dynamically created list of users. Also, it would be good to not have them clash with optional repository local custom hooks, I think symlinking would break that.

Since we currently are upgrading from an old customized 5.x install to 8.11.x I have some time on my hands to look at this. Would you be accepting merge requests?

How about the idea of implementing this using the suggested global custom_hooks directory, but instead of symlinking and potentially clashing with local custom hooks, the global ones are simply also getting called via GitlabCustomHook#call_receive_hook? Filtering out unwanted repositories could happen in the hooks themselves, at the same time keeping configuration somewhat central.

@dirker Sure, we'll entertain any merge request for this. We may also have to put something in the GitLab app to call custom hooks when using the web UI to commit.

mentioned in merge request !89 (closed)

I created !89 (closed) which implements my proposal. Please review and let me know.

It's been 3 weeks since the activity on the MR, I hope this feature isn't loosing momentum. We have several policies that we're trying to implement and I see this feature as very helpful. It would allow us to keep the major policies uniform across all projects.

Looking forward for a fix. Providing a way to manage such hooks via UI will be really awesome.

Mentioned in merge request !93 (closed)

Just ran into this while upgrading to 8.10. Seems like there was something changed on hooks/post-receive and git therefor jumps into my face and won't upgrade until i manually figured out what my predecessor did to this script... The proper way to implement this seems to be a global custom hook. Please add this feature soon! I just hate managing custom patched files ;) Thanks!

I spoke with a prospect for EE yesterday who wanted this. The company's own product interfaces with Gerrit via git (global?) hooks. A a part of moving from Gerrit to GitLab they are looking for that type of functionality in GitLab. They don't want to have to implement the hooks separately for each project. They have a LOT of projects.

Prospect details here: https://na34.salesforce.com/0066100000D1AUo

@xyzzy This is being implemented in either https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/89 or https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/93 - These are waiting a review from @DouweM afaik

Requested https://gitlab.zendesk.com/agent/tickets/47561

@smcgivern Can you look into https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/89 or https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/93 maybe we can keep the preferred implementation open?

I'm really sorry about how long this has taken. Basically, as you can see, we are waiting on some very busy people to review this, and as gitlab-shell rarely has features that on their own are this important, we don't have the best workflow for prioritising this.

I think I prefer !93 (closed) because it doesn't add much more complexity, but the flexibility seems really useful and is implemented in a nice way. I've spoken to @DouweM and @rymai briefly about this, and I think we all agree it's better to make a decision and get something merged. If this turns out to be the wrong decision, we can always change the code - that's the great thing about software!

@dirker thank you for working on this, and for your patience - I'll close !89 (closed) now and get to reviewing the other MR. Hopefully we can still make the merge window for 8.14!

Milestone changed to %8.14

Reassigned to @smcgivern

Mentioned in merge request !89 (closed)

Hmm, actually 8.15 might be a better idea. We'd be merging this very close to the release, and it is quite a significant change. But we will do it!

Milestone changed to %8.15

Mentioned in merge request !111 (merged)

closed via merge request !111 (merged)