fs: improve promise based readFile performance for big files
This significantly reduces the peak memory for the promise based readFile operation by reusing a single memory chunk after each read and strinigifying that chunk immediately.
Refs: https://github.com/nodejs/node/discussions/44239#discussioncomment-3428693
Signed-off-by: Ruben Bridgewater ruben@bridgewater.de
Benchmark with a few runs:
confidence improvement accuracy (*) (**) (***)
fs/readfile-promises.js concurrent=1 len=1024 duration=5 -1.18 % ±7.07% ±9.67% ±13.12%
fs/readfile-promises.js concurrent=1 len=16777216 duration=5 -2.11 % ±3.34% ±4.56% ±6.20%
fs/readfile-promises.js concurrent=1 len=33554432 duration=5 *** 59.48 % ±5.41% ±7.45% ±10.25%
fs/readfile-promises.js concurrent=1 len=4194304 duration=5 *** 9.95 % ±5.10% ±6.93% ±9.32%
fs/readfile-promises.js concurrent=1 len=524288 duration=5 1.88 % ±6.77% ±9.22% ±12.45%
fs/readfile-promises.js concurrent=1 len=8388608 duration=5 ** 6.18 % ±4.19% ±5.70% ±7.69%
fs/readfile-promises.js concurrent=10 len=1024 duration=5 -1.21 % ±8.11% ±11.12% ±15.15%
fs/readfile-promises.js concurrent=10 len=16777216 duration=5 *** 9.72 % ±5.07% ±6.92% ±9.36%
fs/readfile-promises.js concurrent=10 len=33554432 duration=5 *** 26.58 % ±6.29% ±8.56% ±11.56%
fs/readfile-promises.js concurrent=10 len=4194304 duration=5 ** 9.66 % ±5.88% ±8.00% ±10.77%
fs/readfile-promises.js concurrent=10 len=524288 duration=5 1.38 % ±5.49% ±7.47% ±10.07%
fs/readfile-promises.js concurrent=10 len=8388608 duration=5 *** 16.84 % ±4.35% ±5.93% ±8.02%
Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 12 comparisons, you can thus
expect the following amount of false-positive results:
0.60 false positives, when considering a 5% risk acceptance (*, **, ***),
0.12 false positives, when considering a 1% risk acceptance (**, ***),
0.01 false positives, when considering a 0.1% risk acceptance (***)