CI Cache tarballs contain duplicate files causing massive cache files.
For some reason, the tarballs generated by the CI cache contain duplicates files. This is all after only one CI build for a given job (docker rm -v $(docker ps -aq) was run before hand).
First of all, the cache tarball itself is 1.3 GB. (The pre-packed contents are 340 MB, as reported by du -sh).
# find / -name 'cache.tgz' | xargs ls -hl
-rw------- 1 root root 1.3G Nov 24 15:34 /var/lib/docker/volumes/68987288c510a646f34de93bb132f62cd96854dc73200818a084d5c5daee4b1b/_data/group-name/repo-name/job-name/branch-name/cache.tgzWithin the tarball, all the files are duplicated, but not in a consistent manner.
# tar -ztvf /var/lib/docker/volumes/68987288c510a646f34de93bb132f62cd96854dc73200818a084d5c5daee4b1b/_data/group-name/repo-name/job-name/branch-name/cache.tgz | sort -r -k3 -n | grep -vF '.git' | head -n 50
-rw-r--r-- xfs/xfs 2300916 2015-07-27 23:39 bower_components/moment/min/tests.js
-rw-r--r-- xfs/xfs 2300916 2015-07-27 23:39 bower_components/moment/min/tests.js
-rw-r--r-- xfs/xfs 2300916 2015-07-27 23:39 bower_components/moment/min/tests.js
-rw-r--r-- xfs/xfs 2300916 2015-07-27 23:39 bower_components/moment/min/tests.js
-rwxr-xr-x xfs/xfs 1131710 2015-07-27 18:09 node_modules/modernizr/node_modules/requirejs/bin/r.js
-rwxr-xr-x xfs/xfs 1131710 2015-07-27 18:09 node_modules/modernizr/node_modules/requirejs/bin/r.js
-rwxr-xr-x xfs/xfs 1131710 2015-07-27 18:09 node_modules/modernizr/node_modules/requirejs/bin/r.js
-rwxr-xr-x xfs/xfs 1131710 2015-07-27 18:09 node_modules/modernizr/node_modules/requirejs/bin/r.js
-rwxr-xr-x xfs/xfs 1131710 2015-07-27 18:09 node_modules/modernizr/node_modules/requirejs/bin/r.js
-rwxr-xr-x xfs/xfs 1131710 2015-07-27 18:09 node_modules/modernizr/node_modules/requirejs/bin/r.js
-rw-r--r-- xfs/xfs 1070726 2015-11-20 02:13 bower_components/angular/angular.js
-rw-r--r-- xfs/xfs 1070726 2015-11-20 02:13 bower_components/angular/angular.js
-rw-r--r-- xfs/xfs 1070726 2015-11-20 02:13 bower_components/angular/angular.js
-rw-r--r-- xfs/xfs 931402 2015-11-10 00:25 bower_components/zxcvbn/dist/zxcvbn.js.map
-rw-r--r-- xfs/xfs 931402 2015-11-10 00:25 bower_components/zxcvbn/dist/zxcvbn.js.map
-rw-r--r-- xfs/xfs 931402 2015-11-10 00:25 bower_components/zxcvbn/dist/zxcvbn.js.map
-rw-r--r-- xfs/xfs 931402 2015-11-10 00:25 bower_components/zxcvbn/dist/zxcvbn.js.map
-rw-r--r-- xfs/xfs 821322 2015-11-10 00:25 bower_components/zxcvbn/dist/zxcvbn.js
-rw-r--r-- xfs/xfs 821322 2015-11-10 00:25 bower_components/zxcvbn/dist/zxcvbn.js
-rw-r--r-- xfs/xfs 821322 2015-11-10 00:25 bower_components/zxcvbn/dist/zxcvbn.js
-rw-r--r-- xfs/xfs 821322 2015-11-10 00:25 bower_components/zxcvbn/dist/zxcvbn.js
-rw-r--r-- xfs/xfs 721247 2015-11-24 15:31 vendor/fzaninotto/faker/src/Faker/Provider/fr_FR/Text.php
-rw-r--r-- xfs/xfs 721247 2015-11-24 15:31 vendor/fzaninotto/faker/src/Faker/Provider/fr_FR/Text.php
-rw-r--r-- xfs/xfs 721247 2015-11-24 15:31 vendor/fzaninotto/faker/src/Faker/Provider/fr_FR/Text.php
-rw-r--r-- xfs/xfs 721247 2015-11-24 15:31 vendor/fzaninotto/faker/src/Faker/Provider/fr_FR/Text.php
-rw-r--r-- xfs/xfs 721247 2015-11-24 15:31 vendor/fzaninotto/faker/src/Faker/Provider/fr_FR/Text.php
-rw-r--r-- xfs/xfs 721247 2015-11-24 15:31 vendor/fzaninotto/faker/src/Faker/Provider/fr_FR/Text.php
-rw-r--r-- xfs/xfs 721247 2015-11-24 15:31 vendor/fzaninotto/faker/src/Faker/Provider/fr_FR/Text.php
-rw-r--r-- xfs/xfs 721247 2015-11-24 15:31 vendor/fzaninotto/faker/src/Faker/Provider/fr_FR/Text.php
-rw-r--r-- xfs/xfs 682336 2013-08-28 23:36 node_modules/grunt/node_modules/js-yaml/node_modules/esprima/test/test.js
-rw-r--r-- xfs/xfs 682336 2013-08-28 23:36 node_modules/grunt/node_modules/js-yaml/node_modules/esprima/test/test.js
-rw-r--r-- xfs/xfs 682336 2013-08-28 23:36 node_modules/grunt/node_modules/js-yaml/node_modules/esprima/test/test.js
-rw-r--r-- xfs/xfs 682336 2013-08-28 23:36 node_modules/grunt/node_modules/js-yaml/node_modules/esprima/test/test.js
-rw-r--r-- xfs/xfs 682336 2013-08-28 23:36 node_modules/grunt/node_modules/js-yaml/node_modules/esprima/test/test.js
-rw-r--r-- xfs/xfs 682336 2013-08-28 23:36 node_modules/grunt/node_modules/js-yaml/node_modules/esprima/test/test.js
-rw-r--r-- xfs/xfs 682336 2013-08-28 23:36 node_modules/grunt/node_modules/js-yaml/node_modules/esprima/test/test.js
-rw-r--r-- xfs/xfs 682336 2013-08-28 23:36 node_modules/grunt/node_modules/js-yaml/node_modules/esprima/test/test.jsAs you can see, files are duplicated several times. I have seen some files be duplicated over 9 times.
