fs.readFile does not partition reads - threadpool exhaustion
Version: Everything Platform: All Subsystem: FS
Description
The node master, and all previous releases AFAIK, implement fs.readFile
as a call to stat, followed by a request to read the entire file based on the size reported by stat.
Why is this bad? The effect is to place on the libuv threadpool a "giant" read request, occupying the libuv thread until it completes. While readFile certainly requires buffering the entire file contents, it can partition the read into smaller buffers (as is done on other read paths) to avoid threadpool squatting.
If the file is relatively large or stored on a slow medium, reading the entire file in one shot seems particularly harmful, and seems to present a possible DoS vector.
Downsides to partitioning?
I don't think partitioning the read like this raises any additional risk of read-write races on the FS. If the application is concurrently readFile'ing and modifying the file, it will already see funny behavior. Though libuv uses preadv
where available, this doesn't guarantee read atomicity in the presence of concurrent writes.
Related It might be that writeFile has similar behavior. I didn't check.
PR I have a patch that partitions readFile. I'm happy to submit a PR if there's interest.