Skip to content

feature-request - add [lazy] option `parents` to fs.writeFile[Sync], fs.appendFile[Sync], fs.open[Sync]

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

this is proof-of-concept to implement feature-request #33559 (closed). if feedback is positive/viable, then i will proceed with adding documentation, tests, and other checklist items.

motivation

this feature is intended to improve ergonomics/simplify-scripting-tasks when:

  • scaffolding new web-projects
  • creating build-artifacts/coverage-files during ci
  • cloning website with web-crawler
  • uploading files to server

in above tasks, user often has no deterministic-knowledge on directory-structure before file-creation. this feature allows user to lazily create ad-hoc directory-structures as need during file-creation using ergonomic syntax:

fs.writeFileSync(
    "foo/bar/baz/qux.txt",
    "hello world!",
    { parents: true } // will lazily "mkdir --parents" foo/bar/baz as needed
);

performance impact and benchmark

the benchmark (in windows 10) comparing this pr-branch against master-branch shows no-performance-impact on fs.writeFile[Sync] or fs.appendFile[Sync] when option { parents: true } is not used.

when option { parents: true } is enabled:

  • fs.writeFileSync and fs.appendFileSync is ~10% slower at lazy-adhoc-directory-creation (vs eager-determistic-directory-creation)
  • fs.writeFile and fs.appendFile is ~70% slower at lazy-adhoc-directory-creation (vs eager-determistic-directory-creation)

windows benchmark result

the following results should be reproducible by following benchmark instructions at https://github.com/kaizhu256/node/tree/benchmark.fs.writeFile.mkdirRecursive#run-windows-benchmark

async-operation                                         master-branch           pr-branch       performance-impact
30 * 1000 * (fs.writeFile      w/ no-mkdir      )      934 +/-  50 ms      929 +/-  82 ms    no performance-impact
30 * 1000 * (fs.writeFile      w/ fs.mkdir      )      722 +/- 163 ms      722 +/- 159 ms    no performance-impact
30 * 1000 * (fs.writeFile      w/ parents)             ---- ms     1244 +/- 135 ms    72% slower (lazy-parents vs eager-fs.mkdir)

async-operation                                         master-branch           pr-branch       performance-impact
30 * 1000 * (fs.appendFile     w/ no-mkdir      )      983 +/-  62 ms      966 +/-  92 ms    no performance-impact
30 * 1000 * (fs.appendFile     w/ fs.mkdir      )      739 +/- 175 ms      730 +/- 184 ms    no performance-impact
30 * 1000 * (fs.appendFile     w/ parents)             ---- ms     1252 +/- 126 ms    69% slower (lazy-parents vs eager-fs.mkdir)


 sync-operation                                         master-branch           pr-branch       performance-impact
30 * 1000 * (fs.writeFileSync  w/ no-mkdir      )     1008 +/-  42 ms     1007 +/-  45 ms    no performance-impact
30 * 1000 * (fs.writeFileSync  w/ fs.mkdirSync  )     1643 +/- 121 ms     1611 +/- 117 ms    no performance-impact
30 * 1000 * (fs.writeFileSync  w/ parents)             ---- ms     1791 +/-  87 ms     9% slower (lazy-parents vs eager-fs.mkdir)

 sync-operation                                         master-branch           pr-branch       performance-impact
30 * 1000 * (fs.appendFileSync w/ no-mkdir      )      999 +/-  47 ms     1019 +/-  53 ms    no performance-impact
30 * 1000 * (fs.appendFileSync w/ fs.mkdirSync  )     1617 +/- 103 ms     1619 +/- 109 ms    no performance-impact
30 * 1000 * (fs.appendFileSync w/ parents)             ---- ms     1789 +/- 104 ms    11% slower (lazy-parents vs eager-fs.mkdir)

# bikeshed of name mkdirRecursive when scripting the benchmark, i realized the name mkdirRecursive is tedious to type. it also doesn't convey the lazy-nature of this operation, or the -p attribute. am open to other naming suggestions (e.g. { mkdirp: true }) renamed to parents (from unix idiom mkdir --parents)

Merge request reports

Loading