PR-URL: https://github.com/nodejs/node/pull/48513
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Harshitha K P <harshitha014@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
In general, we assume that the tmpdir will provide sufficient space for
most tests. Some tests, however, require hundreds of megabytes or even
gigabytes of space, which often causes them to fail, especially on our
macOS infrastructure. The most recent reliability report contains more
than 20 related CI failures.
This change adds a new function hasEnoughSpace() to the tmpdir module
that uses statfsSync() to guess whether allocating a certain amount of
space within the temporary directory will succeed.
This change also updates the most frequently failing tests to use the
new function such that the relevant parts of the tests are skipped if
tmpdir has insufficient space.
Refs: https://github.com/nodejs/reliability/issues/549
PR-URL: https://github.com/nodejs/node/pull/47767
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Richard Lau <rlau@redhat.com>
PR-URL: https://github.com/nodejs/node/pull/37798
Refs: https://github.com/whatwg/dom/pull/960
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zijian Liu <lxxyxzj@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
PR-URL: https://github.com/nodejs/node/pull/36716
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Zijian Liu <lxxyxzj@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
`test/parallel/test-fs-readfile.js` has a call to
`assert.ifError()` that receives two arguments.
There is no second argument used in `assert.ifError()`.
This PR removes this argument.
PR-URL: https://github.com/nodejs/node/pull/22190
Reviewed-By: George Adams <george.adams@uk.ibm.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jon Moss <me@jonathanmoss.me>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Problem:
Node implements fs.readFile as:
- a call to stat, then
- a C++ -> libuv request to read the entire file using the stat size
Why is this bad?
The effect is to place on the libuv threadpool a potentially-large
read request, occupying the libuv thread until it completes.
While readFile certainly requires buffering the entire file contents,
it can partition the read into smaller buffers
(as is done on other read paths)
along the way to avoid threadpool exhaustion.
If the file is relatively large or stored on a slow medium, reading
the entire file in one shot seems particularly harmful,
and presents a possible DoS vector.
Solution:
Partition the read into multiple smaller requests.
Considerations:
1. Correctness
I don't think partitioning the read like this raises
any additional risk of read-write races on the FS.
If the application is concurrently readFile'ing and modifying the file,
it will already see funny behavior. Though libuv uses preadv where
available, this doesn't guarantee read atomicity in the presence of
concurrent writes.
2. Performance
Downside: Partitioning means that a single large readFile will
require into many "out and back" requests to libuv,
introducing overhead.
Upside: In between each "out and back", other work pending on the
threadpool can take a turn.
In short, although partitioning will slow down a large request,
it will lead to better throughput if the threadpool is handling
more than one type of request.
Fixes: https://github.com/nodejs/node/issues/17047
PR-URL: https://github.com/nodejs/node/pull/17054
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
