Node zopfli no longer maintained
The node-zopfli dependency has not been updated in over 9 months, and currently has no support for Node 8. There's an open pull-request to build against it, but it has been sitting un-merged since June.
As a result, our yarn install
process throws errors when anyone runs it with node 8.x installed on their system:
$ yarn install
yarn install v0.27.5
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
[-/3] ⠐ waiting...
[-/3] ⠐ waiting...
[3/3] ⠐ node-zopfli: https://node-zopfli.s3.amazonaws.com/Release/zopfli-v2.0.2-node-v57-darwin-x64.t
[-/3] ⠐ waiting...
warning Error running install script for optional dependency: "/Users/mike/Projects/gitlab-ce/gitlab/node_modules/node-zopfli: Command failed.\nExit code: 1\nCommand: sh\nArguments: -c node-pre-gyp install --fallback-to-build\nDirectory: /Users/mike/Projects/gitlab-ce/gitlab/node_modules/node-zopfli\nOutput:\nnode-pre-gyp info it worked if it ends with ok\nnode-pre-gyp info using node-pre-gyp@0.6.36\nnode-pre-gyp info using node@8.3.0 | darwin | x64\nnode-pre-gyp info check checked for \"/Users/mike/Projects/gitlab-ce/gitlab/node_modules/node-zopfli/lib/binding/node-v57-darwin-x64/zopfli.node\" (not found)\nnode-pre-gyp http GET https://node-zopfli.s3.amazonaws.com/Release/zopfli-v2.0.2-node-v57-darwin-x64.tar.gz\nnode-pre-gyp http 403 https://node-zopfli.s3.amazonaws.com/Release/zopfli-v2.0.2-node-v57-darwin-x64.tar.gz\nnode-pre-gyp ERR! Tried to download(403): https://node-zopfli.s3.amazonaws.com/Release/zopfli-v2.0.2-node-v57-darwin-x64.tar.gz \nnode-pre-gyp ERR! Pre-built binaries not found for node-zopfli@2.0.2 and node@8.3.0 (node-v57 ABI) (falling back to source compile with node-gyp) \nnode-pre-gyp http 403 status code downloading tarball https://node-zopfli.s3.amazonaws.com/Release/zopfli-v2.0.2-node-v57-darwin-x64.tar.gz \nnode-pre-gyp ERR! build error \nnode-pre-gyp ERR! stack Error: Failed to execute 'node-gyp clean' (Error: spawn node-gyp ENOENT)\nnode-pre-gyp ERR! stack at ChildProcess.<anonymous> (/Users/mike/Projects/gitlab-ce/gitlab/node_modules/node-pre-gyp/lib/util/compile.js:77:29)\nnode-pre-gyp ERR! stack at emitOne (events.js:115:13)\nnode-pre-gyp ERR! stack at ChildProcess.emit (events.js:210:7)\nnode-pre-gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)\nnode-pre-gyp ERR! stack at onErrorNT (internal/child_process.js:374:16)\nnode-pre-gyp ERR! stack at _combinedTickCallback (internal/process/next_tick.js:138:11)\nnode-pre-gyp ERR! stack at process._tickCallback (internal/process/next_tick.js:180:9)\nnode-pre-gyp ERR! System Darwin 16.7.0\nnode-pre-gyp ERR! command \"/usr/local/Cellar/node/8.3.0_1/bin/node\" \"/Users/mike/Projects/gitlab-ce/gitlab/node_modules/node-zopfli/nDone in 12.21s.
Suggested remedies
Option 1: We remove zopfli support and go back to gzip. (This would be easiest to implement but we'll take a small network performance hit)
Option 2: We create a fork of webpack-compression-plugin
that uses a fork of node-zopfli
that compiles against node 8 correctly.
Option 3: We stop using webpack to compress our frontend assets and create a special script to do this instead. We can parallelize this task so machines with multiple CPUs can perform asset compression more quickly (related to #35305 (moved)). We'd have to include instructions to install the zopfli library for various linux distributions in our install-from-source instructions (we could make this an optional step), and we could bake the zopfli binary into our build images ahead of time for omnibus.
Option 3 requires the most work, but I think it would be the best way to do it. We'd also greatly improve build performance since zopfli is very slow and computationally demanding.
/cc @marin