feature-request - add [lazy] option `parents` to fs.writeFile[Sync], fs.appendFile[Sync], fs.open[Sync]
Checklist
-
make -j4 test
(UNIX), orvcbuild test
(Windows) passes -
tests and/or benchmarks are included -
documentation is changed or added -
commit message follows commit guidelines
this is proof-of-concept to implement feature-request #33559 (closed). if feedback is positive/viable, then i will proceed with adding documentation, tests, and other checklist items.
motivation
this feature is intended to improve ergonomics/simplify-scripting-tasks when:
- scaffolding new web-projects
- creating build-artifacts/coverage-files during ci
- cloning website with web-crawler
- uploading files to server
in above tasks, user often has no deterministic-knowledge on directory-structure before file-creation. this feature allows user to lazily create ad-hoc directory-structures as need during file-creation using ergonomic syntax:
fs.writeFileSync(
"foo/bar/baz/qux.txt",
"hello world!",
{ parents: true } // will lazily "mkdir --parents" foo/bar/baz as needed
);
performance impact and benchmark
the benchmark (in windows 10) comparing this pr-branch against master-branch shows no-performance-impact on fs.writeFile[Sync] or fs.appendFile[Sync] when option { parents: true }
is not used.
when option { parents: true }
is enabled:
- fs.writeFileSync and fs.appendFileSync is ~10% slower at lazy-adhoc-directory-creation (vs eager-determistic-directory-creation)
- fs.writeFile and fs.appendFile is ~70% slower at lazy-adhoc-directory-creation (vs eager-determistic-directory-creation)
windows benchmark result
the following results should be reproducible by following benchmark instructions at https://github.com/kaizhu256/node/tree/benchmark.fs.writeFile.mkdirRecursive#run-windows-benchmark
async-operation master-branch pr-branch performance-impact
30 * 1000 * (fs.writeFile w/ no-mkdir ) 934 +/- 50 ms 929 +/- 82 ms no performance-impact
30 * 1000 * (fs.writeFile w/ fs.mkdir ) 722 +/- 163 ms 722 +/- 159 ms no performance-impact
30 * 1000 * (fs.writeFile w/ parents) ---- ms 1244 +/- 135 ms 72% slower (lazy-parents vs eager-fs.mkdir)
async-operation master-branch pr-branch performance-impact
30 * 1000 * (fs.appendFile w/ no-mkdir ) 983 +/- 62 ms 966 +/- 92 ms no performance-impact
30 * 1000 * (fs.appendFile w/ fs.mkdir ) 739 +/- 175 ms 730 +/- 184 ms no performance-impact
30 * 1000 * (fs.appendFile w/ parents) ---- ms 1252 +/- 126 ms 69% slower (lazy-parents vs eager-fs.mkdir)
sync-operation master-branch pr-branch performance-impact
30 * 1000 * (fs.writeFileSync w/ no-mkdir ) 1008 +/- 42 ms 1007 +/- 45 ms no performance-impact
30 * 1000 * (fs.writeFileSync w/ fs.mkdirSync ) 1643 +/- 121 ms 1611 +/- 117 ms no performance-impact
30 * 1000 * (fs.writeFileSync w/ parents) ---- ms 1791 +/- 87 ms 9% slower (lazy-parents vs eager-fs.mkdir)
sync-operation master-branch pr-branch performance-impact
30 * 1000 * (fs.appendFileSync w/ no-mkdir ) 999 +/- 47 ms 1019 +/- 53 ms no performance-impact
30 * 1000 * (fs.appendFileSync w/ fs.mkdirSync ) 1617 +/- 103 ms 1619 +/- 109 ms no performance-impact
30 * 1000 * (fs.appendFileSync w/ parents) ---- ms 1789 +/- 104 ms 11% slower (lazy-parents vs eager-fs.mkdir)
# bikeshed of name
mkdirRecursive
when scripting the benchmark, i realized the name
renamed to mkdirRecursive
is tedious to type. it also doesn't convey the lazy-nature of this operation, or the -p
attribute. am open to other naming suggestions (e.g. { mkdirp: true }
)parents
(from unix idiom mkdir --parents
)