Users lose settings after server termination.

Summary

We have our Gitlab instance running in an Auto-Scaling Group where we only expect 1 instance to be online at a given time. We simply want easy recovery if we lose the single host.

The server is configured to use an RDS instance as an external database, and EFS as an NFS file share configured as described in the High-Availability documentation. We are not using an external Redis instance.

For logins we are using OmniAuth w/ Google Apps. Users are set to not be external if they sign up, and the configuration limits to the domain to ours. It is also set to auto-login.

The issue we are seeing is if the instance terminates and recreates, users's Access permissions don't survive. I'm referring to the settings configured here:

https://gitlab.ceroic.com/admin/users/username/edit

The one's we've noticed specifically are:

• Projects Limit
• Can Create Group
• External User

Interestingly enough Admin does survive. The system's defaults are set correctly and survive the termination as well.

Any idea as to what's being persisted and might not be living through the termination?

Steps to reproduce

• Configure Gitlab to start up within an ASG w/ a Launch Configuration
• Configure Gitlab to use an external database and an NFS share per the high availability documentation
• Configure Gitlab to use OmniAuth with Google, disable sign up, and enable automatic login.
• Configure a user w/ a project limit greater than zero, not external, and permission to create groups.
• Terminate the server and allow the ASG to recreate it.

Expected behavior

The user's settings should persist through the termination / recreate.

Actual behavior

The user will have 0 for Projects Limit, be not able to create a group, and be flagged as an external user.

Relevant logs and/or screenshots

(Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's very hard to read otherwise.)

Output of checks

Results of GitLab application Check

Checking GitLab Shell ...

GitLab Shell version >= 3.4.0 ? ... OK (3.4.0)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
1/1 ... repository is empty
5/3 ... repository is empty
6/4 ... repository is empty
2/5 ... repository is empty
7/6 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /mnt/efs/gitlab-data/home/.ssh/authorized_keys: OK
Send ping to redis server: OK
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Reply by email ...

Reply by email is disabled in config/gitlab.yml

Checking Reply by email ... Finished

Checking LDAP ...

LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab ...

Git configured with autocrlf=input? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory setup correctly? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
projects have namespace: ...
1/1 ... yes
5/3 ... yes
6/4 ... yes
2/5 ... yes
7/6 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.3.1)
Your git bin path is "/opt/gitlab/embedded/bin/git"
Git version >= 2.7.3 ? ... yes (2.7.4)
Active users: 3

Checking GitLab ... Finished

Results of GitLab environment info

System information
System:		Ubuntu 14.04
Current User:	git
Using RVM:	no
Ruby Version:	2.3.1p112
Gem Version:	2.6.6
Bundler Version:2.3.0
Rake Version:	10.5.0
Sidekiq Version:4.1.4

GitLab information
Version:	8.11.5-ee
Revision:	b7548a4
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	9.5.2
URL:		https://gitlab.ceroic.com
HTTP Clone URL:	https://gitlab.ceroic.com/some-group/some-project.git
SSH Clone URL:	git@gitlab.ceroic.com:some-group/some-project.git
Elasticsearch:	no
Geo:		no
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers: google_oauth2

GitLab Shell
Version:	3.4.0
Repository storage paths:
- default: 	/mnt/efs/gitlab-data/git-data/repositories
Hooks:		/opt/gitlab/embedded/service/gitlab-shell/hooks/
Git:		/opt/gitlab/embedded/bin/git

Possible fixes

Determine what data is not being persisted between server terminations.

@gmeans Is the OmniAuth provider configured for Google as an external provider? If so, the values for project limits and external flags will be reset on login.

Added ~157390 label

Yup that was it. Thanks!

I'm not sure if I missed it in the docs, but it wasn't clear to me that gitlab_rails['omniauth_external_providers'] indicated that all users signing up under these OmniAuth providers would be marked external users. I assumed it meant these were the active external to Gitlab providers that were allowed to auth.

I also think it conflicts w/ the Admin area's "New users set to external" setting.

Is it worth an issue or doc update? Or did I just miss it?

@gmeans I seem to remember this being a cause of confusion in the past. I'll take a look into it.

Added ~111707 label

Reassigned to @markglenfletcher

Assignee removed

Reassigned to @markglenfletcher

added docs-todo label

added docs-write and removed docs-todo labels

assigned to @axil

@markglenfletcher all we need is document what you wrote in https://gitlab.com/gitlab-org/gitlab-ee/issues/999#note_15181638?

Yes. Thanks @axil

Great, moving to 9.0.

changed milestone to %9.0

removed assignee

changed milestone to %9.5