Support for statically compressed gzip content-encoding

Related to #12 (closed), this MR enables serving precompressed gzip content in GitLab Pages.

To use it, static files that are to be served compressed must be present at the same location as the uncompressed file, with the extension .gz. The uncompressed file must remain present so that:

• HTTP clients that do not nominate an Accept-Encoding of gzip can still receive the content,
• Range requests continue to work, by serving the sliced uncompressed content (otherwise the daemon would need to recompress on the fly),
• The daemon can auto-detect the difference between the compressed variant and, for example, a .tar.gz file which should not have the Content-Encoding header attached to the response. (Otherwise the browser may decompress and present a .tar file to the user.)

The gzipped content must be placed in the same directory as the uncompressed content. When the latter is a symlink, the gzipped content must be placed at the symlink target directory with the same file basename as the target. For example, if the file index.html is symlinked to content/frontpage.html, then the gzipped content must be located at content/frontpage.html.gz. This is both to co-locate the gzipped content with the uncompressed variant and to simplify the implementation.

Directory index.html files and the /404.html files can also be compressed by having corresponding index.html.gz and /404.html.gz compressed variants.

HTTP HEAD requests continue to work correctly; the Content-Encoding header is also applied as appropriate, so that HTTP clients can know that they may receive compressed content in that case.

I am keen to see this merged and deployed to GitLab Pages so that my (and everyone's) static sites load much faster: HTML, CSS, and JavaScript as you know often compresses with very high ratios. I'm sure GitLab Inc. would enjoy the bandwidth (cost) savings too.

@ayufan, please let me know how this looks to you! :D

mentioned in merge request !27 (merged)

The symlink edge case is interesting. Starting with the following structure:

a/
  foo
  bar -> ../b/bar
b/
  bar

The symlink-unaware user would run gzip against every apparent file, leading to:

a/
  foo
  foo.gz
  bar -> ../b/bar
  bar.gz
b/
  bar
  bar.gz

so we end up with a tree that will support either scheme. It only becomes an issue when a user subsequently modifies a/bar.gz and those changes are not reflected in the HTTP response as it serves b/bar.gz when asked for a/bar.

I think serving a/bar.gz would be marginally better, but it's not a strong preference. Wdyt?

added 4 commits

• 11324848...043be6d2 - 3 commits from branch gitlab-org:master
• 3ee71e54 - Support for statically compressed gzip content-encoding

Compare with previous version

Despite the busy log above, the MR has been updated with a single squashed commit 12fcbe0b.

I thought that force-pushing the source repo would result in a clean update here; unfortunately not and so my apologies.

mentioned in issue #61

added 3 commits

• 37c9d979...043be6d2 - 2 commits from branch gitlab-org:master
• 861eb7ea - Support for statically compressed gzip content-encoding

Compare with previous version

@nick.thomas, anything else I can do to help this MR along?

resolved all discussions

Sorry @jap, I missed the last push!

This LGTM, but I'd like @ayufan to +1 it before merge.

assigned to @ayufan

It looks LGTM too 👍

assigned to @nick.thomas

OK, merging. Thanks @jpap !

merged

mentioned in commit 838c75f0

Fantastic @nick.thomas! When might we expect this to go live on gitlab.com?

@jpap we may be able to get it into 9.2, in which case it would go live on the first 9.2 RC - the date for that varies, but no later than the 22nd (of course!)

However, we may miss 9.2.

Fingers crossed you make 9.2! I can't wait to see this go live and use it to speed initial display of our JS-based webapps given the huge ratios typical of JS bundles.

Thanks for your help pushing this through! :D

mentioned in issue #12 (closed)

mentioned in merge request !8 (closed)

mentioned in issue #75 (closed)