Configure Apache for new gitlab-workhorse

See gitlab-org/gitlab-ce!2073

Milestone changed to 8.3

Here's the config I am using right now. I only did some superficial testing.

  #Forward everything to the workhorse                                                                                                                                  
  ProxyPass / http://127.0.0.1:8181/
  RequestHeader set X_FORWARDED_PROTO 'https'
  RequestHeader set X-Forwarded-Ssl on

  # needed for downloading attachments                                                                                                                                  
  DocumentRoot /opt/gitlab/embedded/service/gitlab-rails/public/
  <Directory /opt/gitlab/embedded/service/gitlab-rails/public/>
    Require all granted
  </Directory>

  #Set up apache error documents, if back end goes down (i.e. 503 error) then a maintenance/deploy page is thrown up.                                                   
  ErrorDocument 404 /404.html
  ErrorDocument 422 /422.html
  ErrorDocument 500 /500.html
  ErrorDocument 503 /deploy.html

@ralf could you open a merge request with that?

I'm working on updating all configs. Should be up in a couple of hours.

I haven't updated to 8.3.x because I've been waiting for the apache configs to be updated here. @axil I see you did not yet push the updated configs. Should I wait for it or should I use the snipped above? Or is there a WIP branch from where I can fetch and test the updated configs? Is there anything I can do to help get this Ticket closed?

Sorry, I got busy with other stuff. I'm trying this right now on my server and will push in a new MR once finished.

I have been trying to get this to work for the past hour with no luck. @ralf can you post the whole apache config you've been using? Maybe I'm missing something.

@axil If its any help, I'm trying with this (apache 2.2, source install) and things appear to be working, although I have not tested thoroughly:

<VirtualHost *:443>
  SSLEngine on
  #strong encryption ciphers only
  #see ciphers(1) http://www.openssl.org/docs/apps/ciphers.html
  SSLProtocol all -SSLv2
  SSLHonorCipherOrder on
  SSLCipherSuite "ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS"
  Header add Strict-Transport-Security: "max-age=15768000;includeSubdomains"
  SSLCompression Off
  SSLCertificateFile /etc/httpd/ssl.crt/YOUR_SERVER_FQDN.crt
  SSLCertificateKeyFile /etc/httpd/ssl.key/YOUR_SERVER_FQDN.key
  SSLCACertificateFile /etc/httpd/ssl.crt/your-ca.crt

  ServerName YOUR_SERVER_FQDN
  ServerSignature Off

  ProxyPreserveHost On

  # Ensure that encoded slashes are not decoded but left in their encoded state.
  # http://doc.gitlab.com/ce/api/projects.html#get-single-project
  AllowEncodedSlashes NoDecode

  <Location />
    Order deny,allow
    Allow from all

    #Allow forwarding to gitlab-workhorse
    ProxyPassReverse http://127.0.0.1:8181
    #Allow forwarding to GitLab Rails app (Unicorn)
    ProxyPassReverse http://127.0.0.1:8080
    ProxyPassReverse http://YOUR_SERVER_FQDN/
  </Location>

  # Apache equivalent of nginx try files
  # http://serverfault.com/questions/290784/what-is-apaches-equivalent-of-nginxs-try-files
  # http://stackoverflow.com/questions/10954516/apache2-proxypass-for-rails-app-gitlab
  RewriteEngine on

  #Forward these requests to gitlab-workhorse
  RewriteRule .* http://127.0.0.1:8181%{REQUEST_URI} [P,QSA,NE]

  RequestHeader set X_FORWARDED_PROTO 'https'
  RequestHeader set X-Forwarded-Ssl on
 # needed for downloading attachments
  DocumentRoot /home/git/gitlab/public

  #Set up apache error documents, if back end goes down (i.e. 503 error) then a maintenance/deploy page is thrown up.
  ErrorDocument 404 /404.html
  ErrorDocument 422 /422.html
  ErrorDocument 500 /500.html
  ErrorDocument 503 /deploy.html

  # It is assumed that the log directory is in /var/log/httpd.
  # For Debian distributions you might want to change this to
  # /var/log/apache2.
  LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common_forwarded
  ErrorLog /var/log/httpd/logs/YOUR_SERVER_FQDN_error.log
  CustomLog /var/log/httpd/logs/YOUR_SERVER_FQDN_forwarded.log common_forwarded
  CustomLog /var/log/httpd/logs/YOUR_SERVER_FQDN_access.log combined env=!dontlog
  CustomLog /var/log/httpd/logs/YOUR_SERVER_FQDN.log combined

</VirtualHost>

mentioned in merge request !50 (merged)

With the recent commits pushed to master I think this Issue is now solved. I finally updated my Server from 8.2.3 to 8.4.3 using these apache configs.

/cc @aneumann91

Status changed to closed

Thanks @p-schneider :)

The ErrorDocument pages are non-functional in this updated Apache configuration, because apache tries to serve them via the proxy, but of course the reason for a 503 error in the first place is that the proxy isn't working. So you get the ugly, generic apache page:

Service Unavailable

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Additionally, a 503 Service Unavailable error was encountered while trying to use an ErrorDocument to handle the request.

instead of the much nicer deploy.html page when workhorse is stopped (e.g. for an upgrade).

This can be fixed by adding a RewriteCond to let Apache handle the error/deploy files:

RewriteCond %{REQUEST_URI} !^/(?:500\.html|deploy\.html|static\.css|logo\.svg)$

immediately before the RewriteRule. Additionally the ErrorDocument 404/422 should be removed, since there is no way Apache is ever going to generate them while proxying everything as in the updated configuration. (I'm not entirely sure about the 500, but I think there are some apache misconfiguration cases that might still cause it before trying to proxy the request).