SSL errors with the Docker container registry
When setting up the Docker container registry with the GDK, there doesn't seem to be a proper way to configure the SSL certificates. I ran into the following error when visiting the container registry page on a project(localhost:3000/namespace/some-project/container_registry
).
Faraday::SSLError in Projects::Registry::RepositoriesController#index
SSL_connect returned=1 errno=0 state=error: certificate verify failed
See the "Further workaround" section below for more info on how we worked around it.
Some relevant documentation
- https://docs.gitlab.com/ce/administration/container_registry.html
- https://docs.gitlab.com/ce/user/project/container_registry.html
Using a secure HTTPS/SSL registry
Generate certs (via https://serverfault.com/a/845788/182342)
openssl req \
-newkey rsa:2048 \
-x509 \
-nodes \
-keyout /Users/eric/my-ce-docker-registry/my-certs/localhost.key \
-new \
-out /Users/eric/my-ce-docker-registry/my-certs/localhost.crt \
-subj /CN=localhost \
-reqexts SAN \
-extensions SAN \
-config <(cat /System/Library/OpenSSL/openssl.cnf \
<(printf '[SAN]\nsubjectAltName=DNS:localhost')) \
-sha256 \
-days 3650
Using Docker for Mac 17.06.2-ce-mac27. After this command you should be able to visit https://localhost:5063/v2/_catalog
and get {"repositories":[]}
docker run -d \
-p 5063:5000 \
--restart=always \
--name registry-ce \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/localhost.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/localhost.key \
-v /Users/eric/my-ce-docker-registry:/var/lib/registry \
-v /Users/eric/my-ce-docker-registry/my-certs/:/certs \
registry:2
$gdk/gitlab/config/gitlab.yml
registry:
enabled: true
host: localhost
port: 5063
api_url: https://localhost:5063/ # internal address to the registry, will be used by GitLab to directly communicate with API
key: /Users/eric/my-ce-docker-registry/my-certs/localhost.key
path: Users/eric/my-ce-docker-registry/
# not sure what the issuer would be???
# issuer: gitlab-issuer
Visiting the container registry page on a project(localhost:3000/namespace/some-project/container_registry
) will show the Faraday::SSLError
now.
Further workaround
@stanhu mentioned this workaround which mimics how we setup things in omnibus.
config/initializers/1_settings.rb
# Works
ENV['SSL_CERT_FILE'] = "/Users/eric/my-ce-docker-registry/my-certs/localhost.crt"
# or ...
# Does not work
ENV['SSL_CERT_DIR'] = "/Users/eric/my-ce-docker-registry/my-certs/"
@stanhu also linked some more documentation about Faraday and SSL certs. I was unable to use the OPENSSLDIR
(from openssl version -a
) because /System/Library/OpenSSL/certs
is protected with macOS "System Integrity Protection" which means I can't copy anything into it even with sudo
(perhaps there is some special command to get things in there
I also tried the following because it seemed related but didn't seem to make a difference, https://docs.gitlab.com/ce/install/installation.html#using-https
$gdk/gitlab-shell/config.yml
http_settings:
// ...
ca_file: /Users/eric/my-ce-docker-registry/my-certs/localhost.crt
// or
ca_path: /Users/eric/my-ce-docker-registry/my-certs/
Is there a more concrete way to get things working?
Potential Solutions
We could add $gdk/gitlab/config/gitlab.yml -> registry -> cert
that would setup the SSL_CERT_FILE
environment variable manually.
It's not quite clear to me how we handle multiple certs (say one for the main GitLab domain and another for the registry) if you can only define a single cert via SSL_CERT_FILE
(SSL_CERT_DIR
didn't work for me). Omnibus looks to have a mechanism to handle multiple via /etc/gitlab/trusted-certs/
.
Using an Insecure registry
Update: See https://gitlab.com/snippets/73109#setup-the-container-registry for a full correct way to set this up.
I tried using an insecure(http) registry but GitLab throws an error about the key
missing from the config.
TypeError in Projects::Registry::RepositoriesController#index
no implicit conversion of nil into String
def key_data
-> @key_data ||= File.read(key_file)
end
Setup
docker run -d \
-p 5063:5000 \
--restart=always \
--name registry-ce \
registry:2
Configure the Docker Daemon to allow pushing to the insecure registry. With Docker for Mac, from the menubar icon -> Preference -> Daemon -> Insecure Registries -> Add http://localhost:5063
-> Click the "Apply & Restart" button
$gdk/gitlab/config/gitlab.yml
registry:
enabled: true
host: localhost
port: 5063
api_url: http://localhost:5063/ # internal address to the registry, will be used by GitLab to directly communicate with API
Reference, https://gitlab.slack.com/archives/C02PF508L/p1506031313000134