lib: port rimraf to core as shutil.rmtree
The @nodejs/tooling group has been working on porting rimraf into Node.js Core: https://github.com/nodejs/tooling/issues/13. We've spoken with @isaacs and this is done with his cooperation: https://github.com/iansu/node/pull/1#issuecomment-489389946. We've chosen to implement this in a new module called shutil
rather than adding it to the existing fs.rmdir
code. There seems to be interest in this as a similar PR was recently opened: #28171.
1. Why?
rimraf
is one of the most popular third-party modules and is used in millions of projects. The Tooling Group thinks this kind of commonly used module should be included in Node.js Core and not require a third-party package.
recursive
option to fs.rmdir
?
2. Why not add a By adding a flag rather than a new function feature detection becomes difficult. People have requested a way to detect the recursive option of mkdir
for example.
rimraf
directly?
3. Why port The rimraf
code contains a number of workarounds for bugs and edge cases in platforms like Windows and sunos. These are the result of many issues and bugfixes over the years and we don't want to lose this functionality.
shutil
?
4. Why The name itself comes from Python. Something like rimraf
is not a filesystem primitive, instead it's an algorithm that is built on top of filesystem primitives. On other platforms these kind of utilities are often grouped under a "shell" namespace, and the Tooling team can imagine several additional utilities added to this module in the future.
5. Why is this not implemented in C++?
Like the recursive mkdir operation, recursive rmdir is not atomic. Unlike recursive mkdir, the number of operations is not bounded by the directory depth; instead, it's one operation for each file and directory encountered. Recursive mkdir cannot run its operations in parallel, either; it must create directories in serial, in descending order.
Recursive rmdir, on the other hand, would benefit from removing files in parallel--userland rimraf takes this strategy to remove files quickly as libuv's fs thread pool can handle it. There's no precedent, AFAIK, for running batch operations across multiple libuv threads in Node.js core. The problem is compounded by the necessity to implement a synchronous version of the API; consider execSync's implementation.
Several attempts have been made at implementing recursive rmdir in C++, but none so far have executed these operations using multiple threads. Because of this, a serial C++ implementation is significantly slower (@boneskull benchmarked his own implementation against userland rimraf) than a parallel, JS implementation.
In addition, a C++ implementation would need to consider the workarounds for various filesystems to provide a builtin on-par with userland rimraf. So far, no C++ implementation has attempted this.
Checklist
-
make -j4 test
(UNIX), orvcbuild test
(Windows) passes -
tests and/or benchmarks are included -
documentation is changed or added -
commit message follows commit guidelines -
add rimraf
license to combined license