Built-in NGINX does not get SSL crt updates on a gitlab-ctl reconfigure
Summary
NGINX does not update certificates after a gitlab-ctl reconfigure, if there was a functional certificate already there.
Steps to reproduce
(Mentions of the 'NGINX' server in this report are reffering to the 'baked in' or included NGINX server that comes with gitlab CE.)
- Configure Gitlab for SSL
- Place the SSL key in the appropriate .key file
- Place JUST the server certificate in the appropriate .crt file
- Run gitlab-ctl reconfigure
- Run openssl s_client -showcerts -connect your.hostname.net:443 AND keep the results handy
THEN
- Add the intermediate certificates to the appropriate .crt file
- Run gitlab-ctl reconfigure
- Run openssl s_client -showcerts -connect your.hostname.net:443 and compare the results to the earlier run.
- Reboot the server (or restart the NGINX server manually)
- Run openssl s_client -showcerts -connect your.hostname.net:443 NOW the updates to the .crt file are observed in the NGINX server.
What is the current bug behavior?
Changes to the .crt file are not reflected in the NGINX server after a gitlab-ctl reconfigure
What is the expected correct behavior?
Changes to the .crt file should update the NGINX server after a gitlab-ctl reconfigure, not requiring a server restart, or other service restart.
Relevant logs and/or screenshots
The changes to the .crt file are not being detected, therefore they are not being logged.
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Ubuntu 16.04 Current User: git Using RVM: no Ruby Version: 2.3.5p376 Gem Version: 2.6.6 Bundler Version:1.13.7 Rake Version: 12.0.0 Redis Version: 3.2.5 Git Version: 2.13.5 Sidekiq Version:5.0.4 Go Version: unknown
GitLab information Version: 9.5.5 Revision: 3cf5eba Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: https://gitlab.mycorp.net HTTP Clone URL: https://gitlab.mycorp.net/some-group/some-project.git SSH Clone URL: git@gitlab.mycorp.net:some-group/some-project.git Using LDAP: yes Using Omniauth: no
GitLab Shell Version: 5.8.0 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
Checking GitLab Shell ...
GitLab Shell version >= 5.8.0 ? ... OK (5.8.0) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 2/1 ... ok 2/3 ... ok 6/4 ... ok 2/6 ... ok 2/7 ... ok 2/8 ... repository is empty 2/9 ... ok 8/10 ... ok 2/11 ... ok 11/12 ... ok 9/13 ... ok 2/14 ... ok 2/15 ... ok Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Access to /var/opt/gitlab/.ssh/authorized_keys: OK Send ping to redis server: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
Reply by email is disabled in config/gitlab.yml
Checking Reply by email ... Finished
Checking LDAP ...
Server: ldapmain LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results) HAHA, no. I'm not sharing my LDAP users and structure outside of my org :) Also, the issue is not related to LDAP.
Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 2/1 ... yes 2/3 ... yes 6/4 ... yes 2/6 ... yes 2/7 ... yes 2/8 ... yes 2/9 ... yes 8/10 ... yes 2/11 ... yes 11/12 ... yes 9/13 ... yes 2/14 ... yes 2/15 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.3 ? ... yes (2.3.5) Git version >= 2.7.3 ? ... yes (2.13.5) Active users: ... 12
Checking GitLab ... Finished
Possible fixes
I'm not familiar with Chef, but I suspect a check of the last modified time of the .crt and .key files, compared to the last time gitlab-ctl reconfigure was ran, would be enough to determine 'hey, we should restart NGINX because these files were changed'. I'd think something along those lines would resolve this issue.