No entry in project_authorizations created if project is created for user by admin via API

Summary

I use GitLab as homework submission system and create empty projects per API for the students. Recently the students didn't see their new Projects in the dashboard anymore and had other rights related issues. After some digging i figured that the project_authorizations table was missing entries. Manually creating them seemed to help

Steps to reproduce

As a Admin, create a project for a user via API ( into the user's namespace )

What is the current bug behavior?

User can't see his project in his dashboard User has no access to his project via known link either

What is the expected correct behavior?

User should be able to see and access his projects

Relevant logs and/or screenshots

(Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's very hard to read otherwise.)

Results of GitLab environment info

System information
System:         Debian 8.7
Current User:   git
Using RVM:      no
Ruby Version:   2.1.5p273
Gem Version:    2.2.2
Bundler Version:1.13.7
Rake Version:   10.5.0
Sidekiq Version:4.2.7

GitLab information
Version:        8.16.7
Revision:       eafc896de9
Directory:      /home/git/gitlab
DB Adapter:     postgresql
URL:            https://student.cgv.tugraz.at
HTTP Clone URL: https://student.cgv.tugraz.at/some-group/some-project.git
SSH Clone URL:  git@student.cgv.tugraz.at:some-group/some-project.git
Using LDAP:     yes
Using Omniauth: yes
Omniauth Providers: saml

GitLab Shell
Version:        4.1.1
Repository storage paths:
- default:      /home/git/repositories/
Hooks:          /home/git/gitlab-shell/hooks/
Git:            /usr/bin/git

Results of GitLab application Check

Checking GitLab Shell ...

GitLab Shell version >= 4.1.1 ? ... OK (4.1.1)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ... 
<cut>
lots of ...ok, some repository is empty
i don't see the point in pasting several thousand lines here
</cut>
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /home/git/.ssh/authorized_keys: OK
Send ping to redis server: OK
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Reply by email ...

Reply by email is disabled in config/gitlab.yml

Checking Reply by email ... Finished

Checking LDAP ...

Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
<cut>
Sorry - i don't think it is ok to publish our account database to the public
</cut>
Checking LDAP ... Finished

Checking GitLab ...

Git configured with autocrlf=input? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory setup correctly? ... no
  Try fixing it:
  sudo chmod 700 /home/git/gitlab/public/uploads
  For more information see:
  doc/install/installation.md in section "GitLab"
  Please fix the error above and rerun the checks.
Init script exists? ... yes
Init script up-to-date? ... yes
projects have namespace: ... 
<cut>
lots of ... yes
Again i don't see the point in pasting thousands of lines
</cut>
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.1.5)
Your git bin path is "/usr/bin/git"
Git version >= 2.7.3 ? ... yes (2.11.0)
Active users: 856

Checking GitLab ... Finished

(we will only investigate if the tests are passing)

Possible fixes

It seems to be possible to fix the problem by adding missing entries to various tables:

-- Add Project owners as Masters to their own Project if they are not yet allready
INSERT INTO members (access_level,source_id,source_type,user_id,notification_level,type,created_at,updated_at,created_by_id)
SELECT 40, p.id, 'Project', n.owner_id, 3, 'ProjectMember', p.created_at, p.created_at, n.owner_id
FROM projects p
INNER JOIN course_config c ON c.creator_id=p.creator_id
INNER JOIN namespaces n ON n.id=p.namespace_id AND n.type IS NULL
LEFT JOIN members m ON m.source_id=p.id AND m.source_type='Project' AND n.owner_id=m.user_id
WHERE m.id IS NULL;

-- Insert missing Entries in Cache Table
INSERT INTO project_authorizations (user_id,project_id,access_level)
SELECT m.user_id,m.source_id,m.access_level
FROM members m
LEFT JOIN project_authorizations a ON a.user_id=m.user_id and a.project_id=m.source_id
WHERE m.source_type='Project' and a.project_id IS NULL;

/cc: @ahmadsherif

Update: I figured that also entries for Group Projects were missing, so i had to add the following query to my database fixes:

INSERT INTO project_authorizations (user_id,project_id,access_level)
SELECT m.user_id,p.id as project_id,m.access_level
FROM members m
INNER JOIN projects p ON p.namespace_id = m.source_id
INNER JOIN namespaces n ON p.namespace_id = n.id
LEFT JOIN project_authorizations a ON a.user_id=m.user_id and a.project_id=p.id
WHERE m.source_type='Namespace' and n.type='Group' and a.project_id IS NULL;

I run that stuff now on a hourly cronjob as well as on a global project creation hook.

added ~480950 label

@DouweM @mydigitalself This seems important to fix.

changed milestone to %9.2

/cc @yorickpeterse

@worf What is the exact API endpoint you're using?

@DouweM : /api/v3/projects?name= ... &path= ... &namespace_id= ... &description= ...

POST, And in in the headers i set the PRIVATE-TOKEN (of the Admin)

I suspect that this API doesn't end up calling the service class used for populating project_authorizations. This is a bit weird since creating members should trigger the after_commit hook that takes care of this.

@yorickpeterse : to be honest: at the moment i'm not sure if actually the members are created correctly and project_authorizations isn't populated as a result.

After submitting the bug and sleeping over it i remember that several months ago i had some unexpected behaviour and realized that in some situations the user was not considered owner of his own projects. So this all might be connected.

I'll do more tests ...

@worf any update on this? Trying to get a better sense of priority on it.

added ~65372 label

added Deliverable label

assigned to @to1ne

mentioned in merge request !10910 (merged)

closed via merge request !10910 (merged)

mentioned in commit 39ff8099