Notification recipients are not calculated correctly for projects in subgroups

Zendesk: https://gitlab.zendesk.com/agent/tickets/80117

To start with, a top-level group (let's call it 'Group1') has a user with global notification settings set to 'Watch'. Presumably they should get notifications for all events in the group, including in subgroups and projects at any level.

If an issue is created in a project at the top-level - Group1/Project1, the notification is sent to the user as expected.

However, if an issue is created in a project a few levels deep - Group1/SubGroup/Project2, no notification is sent unless the user is explicitly mentioned.

I believe the bug is in NotificationRecipientService. In this class there are several places where we gather user IDs like this:

    user_ids += user_ids_notifiable_on(project, :custom, action)
    user_ids += user_ids_notifiable_on(project.group, :custom, action)

This will gather watchers and 'notifiable users' on the project itself and the project's group. However, a user that has notification level 'Watch' only appears as a notifiable user on the top-level group ('Group1'). I confirmed this case on the customer's instance using the Rails console. I manually ran the following query where the resource was project, project.group and project.group.parent. Only the project.group.parent revealed the user we expected to get notifications.

<resource>.notification_settings.where(level: NotificationSetting.levels[:global])

cc/ @mydigitalself @smcgivern

@dblessing I don't think we've ever explicitly supported this. If there's an efficient way of getting someone's notifications for all parent groups up the chain, I guess we could do it, though.

cc @jneen

And cc @victorwu - I think this is more Discussion than Platform .

changed milestone to %Next 3-6 months

I agree we've probably never supported this correctly. It was likely an oversight when nested groups were implemented. I'd call it a bug all the same since it's a departure from expected behavior.

Is there someone with experience with nested groups that we could tag in?

@dblessing isn't this a dupe of https://gitlab.com/gitlab-org/gitlab-ce/issues/30549 ?

Looks like it is. I really wouldn't want to do the recursive solution though

We were recently bit by this (in EE) would be great to get fixed.