Docker pull from GitLab Docker image registry keep retrying and finally fails
Summary
Docker pull from GitLab Docker image registry keep retrying and finally fails:
Steps to reproduce
Unfortunately this occurs only in certain images in our private repository, can't give steps. However, similar issue has been reported for example here: https://stackoverflow.com/questions/44370253/gitlab-docker-registry-pull-failed-eof
Example Project
N/A
What is the current bug behavior?
[root@default ~]# docker pull <imagename>
<version>: Pulling from <imagename>
6d987f6f4279: Already exists
b9e06d6d2682: Already exists
d05c7e9bd1ea: Already exists
577cc606431d: Already exists
9bf0ebb2d797: Pull complete
bb1118770077: Pull complete
34d2d657b17a: Extracting [> ] 196.6kB/19.2MB
6f98d9794b23: Download complete
6ea60500000f: Download complete
d9c7d3d26d5c: Download complete
7fbd6a14ebbf: Download complete
794e06bcda0d: Download complete
f29e04507b03: Retrying in 4 seconds
34ce04506f2f: Download complete
95716a8c354d: Download complete
176bfa4f51b0: Download complete
ddc528d5220d: Download complete
83c3f5c0024e: Retrying in 4 seconds
643dc3657c6a: Download complete
After some retries pull interrupts and "unexpected EOF" comes out.
What is the expected correct behavior?
Docker pull would work properly.
Relevant logs and/or screenshots
Available in bug behavior description.
Output of checks
N/A
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Current User: git Using RVM: no Ruby Version: 2.3.5p376 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.0.0 Redis Version: 3.2.5 Git Version: 2.13.5 Sidekiq Version:5.0.4 Go Version: unknown
GitLab information Version: 10.0.2 Revision: 06a5a33 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: https://epp.edelkey.net:9443 HTTP Clone URL: https://epp.edelkey.net:9443/some-group/some-project.git SSH Clone URL: git@epp.edelkey.net:some-group/some-project.git Using LDAP: no Using Omniauth: no
GitLab Shell Version: 5.9.0 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)
Possible fixes
Above mentioned https://stackoverflow.com/questions/44370253/gitlab-docker-registry-pull-failed-eof was proposing fix:
Problem solved by adding proxy_max_temp_file_size 0; inside nginx virtual host configuration!
However, this was working in our case. But, the setting of proxy bufferingoff was working in our case:
proxy_buffering off;