Caching is very slow for Node.JS projects
Hi, I've tried now several cache configurations to speed up the builds of my Node projects but without luck. Here is my current CI configuration
image: node:latest
cache:
key: "$CI_BUILD_REF_NAME"
paths:
- node_modules/
build:
script:
- npm install
- npm run build
test:
script:
- npm install
- npm run test
In every build step the cache.zip
gets extracted at the start of the build step and repacked at the end of the build step which consumes a significant amount of time. Actually when there are many installed packages a fresh npm install
is faster than using the cache which sound actually pretty strange.
Is there some way to avoid zipping the node_modules
folder and copy it as whole into the cache instead? I think this would save quite a lot of time. Actually zipping the node modules folder isn't a good idea anyhow as all file permissions are getting lost...
Of course if the files are getting uploaded into a s3 storage the whole folder needs to be archived (I guess) but then in case of a node modules folder a tar.gz
archive would make more sense (and is maybe faster).
I.e. ideally the Gitlab runner would allow to:
- Disable archiving the cache in case no s3 storage is defined.
- If disc space is critical then it should be possible to define in the runner a "force archive cache" flag.
- Define the format of the archive via the gitlab-ci YAML.
- If no s3 storage is defined or "force archive cache" is set this rule should be ignored.
Especially since yarn is available a fresh install is now much faster than using the current cache...
Thanks a lot!