Commit Graph

38 Commits

Author SHA1 Message Date
Robert Nagy
bdcb7389ed
stream: fix pipeline pump
Refs: https://github.com/nodejs/node/issues/39005

PR-URL: https://github.com/nodejs/node/pull/39006
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2021-06-14 08:13:12 -07:00
Joyee Cheung
a4ea9fc48e
lib: remove unnecessary lazy loads
Now that more modules are included in the snapshot, it's not
necessary to lazy load them anymore

PR-URL: https://github.com/nodejs/node/pull/38737
Refs: https://github.com/nodejs/node/issues/35711
Reviewed-By: Anna Henningsen <anna@addaleax.net>
2021-05-27 21:46:48 +08:00
Antoine du Hamel
ee9e2a2eb6 lib: revert primordials in a hot path
Evidence has shown that use of primordials have sometimes an impact of
performance. This commit reverts the changes who are most likely to be
responsible for performance regression in the HTTP response path.

PR-URL: https://github.com/nodejs/node/pull/38248
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
2021-04-19 10:23:01 +02:00
Rich Trott
1e34df139c stream: move duplicated code to an internal module
Create a utils module for isIterable(), isReadable(), and isStream().

PR-URL: https://github.com/nodejs/node/pull/37508
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2021-02-28 05:07:26 -08:00
Rich Trott
b461a1b125 doc,lib: prepare for stricter multi-line array linting
We're about to turn on a requirement for dangling commas.

PR-URL: https://github.com/nodejs/node/pull/37088
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2021-02-19 09:55:35 -08:00
Antoine du Hamel
419686cdfb stream: refactor to use more primordials
PR-URL: https://github.com/nodejs/node/pull/36346
Reviewed-By: Zijian Liu <lxxyxzj@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
2021-02-01 15:18:25 +01:00
ZiJian Liu
b00bb01db9 lib: refactor to use validateCallback
PR-URL: https://github.com/nodejs/node/pull/36609
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2020-12-29 15:48:57 +00:00
Matteo Collina
e57d8af7e2 stream,zlib: do not use _stream_* anymore
PR-URL: https://github.com/nodejs/node/pull/36618
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2020-12-29 01:47:57 +01:00
ZiJian Liu
6837a6de97 stream: accept iterable as a valid first argument
Fixes: https://github.com/nodejs/node/issues/36437

PR-URL: https://github.com/nodejs/node/pull/36479
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Yongsheng Zhang <zyszys98@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2020-12-15 13:40:40 +00:00
Gil Pedersen
3c38445cc8 stream: fix thrown object reference
PR-URL: https://github.com/nodejs/node/pull/36065
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2020-11-10 16:34:35 +00:00
Robert Nagy
dd0f8f18c2 stream: writableNeedDrain
Don't write to a stream which already has a full buffer.

Fixes: https://github.com/nodejs/node/issues/35341

PR-URL: https://github.com/nodejs/node/pull/35348
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2020-11-10 10:44:16 +01:00
Antoine du Hamel
ad98cf04f3 stream: remove isPromise utility function
The function was not checking if the parameter was actually a Promise
instance, but if it has a `then` method. Removing the utility function
in favor of a clearer `typeof` check, handling the case when the
thenable throws if then method is accessed more than once.

PR-URL: https://github.com/nodejs/node/pull/35925
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2020-11-09 06:17:29 +00:00
Matteo Collina
9c62e0e384 stream: move to internal/streams
Move all the streams constructors to internal/streams
and avoid a circular dependencies between the modules.

See: https://github.com/nodejs/readable-stream/issues/348

PR-URL: https://github.com/nodejs/node/pull/35239
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
2020-09-21 08:23:06 +00:00
Robert Nagy
08e8997d54 stream: simpler and faster Readable async iterator
Reimplement as an async generator instead of a custom
iterator class.

PR-URL: https://github.com/nodejs/node/pull/34035
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2020-07-17 09:06:27 +02:00
Luigi Pinca
b533fb3508 tools: enable no-else-return lint rule
Refs: https://github.com/nodejs/node/pull/32644
Refs: https://github.com/nodejs/node/pull/32662

PR-URL: https://github.com/nodejs/node/pull/32667
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2020-05-16 06:42:16 +02:00
Robert Nagy
6419e59889 stream: pipeline should only destroy un-finished streams
This PR logically reverts https://github.com/nodejs/node/pull/31940
which has caused lots of unnecessary breakage in the ecosystem.

This PR also aligns better with the actual documented behavior:

`stream.pipeline()` will call `stream.destroy(err)` on all streams
except:
  * `Readable` streams which have emitted `'end'` or `'close'`.
  * `Writable` streams which have emitted `'finish'` or `'close'`.

The behavior introduced in https://github.com/nodejs/node/pull/31940
was much more aggressive in terms of destroying streams. This was
good for avoiding potential resources leaks however breaks some
common assumputions in legacy streams.

Furthermore, it makes the code simpler and removes some hacks.

Fixes: https://github.com/nodejs/node/issues/32954
Fixes: https://github.com/nodejs/node/issues/32955

PR-URL: https://github.com/nodejs/node/pull/32968
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Mathias Buus <mathiasbuus@gmail.com>
2020-04-23 19:50:29 +02:00
Robert Nagy
98b6b2d840 stream: complete pipeline with stdio
stdio (stderr & stdout) should for compatibility
reasons not be closed/end():ed. However, this
causes pipeline with a stdio destination to
never finish. This commit fixes this issue at
a performance cost.

Refs: https://github.com/nodejs/node/issues/7606

Fixes: https://github.com/nodejs/node/issues/32363

PR-URL: https://github.com/nodejs/node/pull/32373
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2020-04-06 21:48:25 +02:00
Robert Nagy
05f1df5200 stream: fix pipeline with dest in objectMode
pipeline did not support destination with generator
that does not return strings or buffers.

PR-URL: https://github.com/nodejs/node/pull/32414
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2020-03-25 14:46:10 +01:00
Robert Nagy
2bfb3403ed
stream: avoid destroying writable source
User might still want to be able to use the writable side
of src. This is in the case where e.g. the Duplex input
is not directly connected to its output. Such a case could
happen when the Duplex is reading from a socket and then echos
the data back on the same socket.

PR-URL: https://github.com/nodejs/node/pull/32198
Refs: 4d93e105bf (commitcomment-37751035)
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2020-03-11 20:57:42 -04:00
Robert Nagy
f8f20892e9
stream: avoid destroying http1 objects
http1 objects are coupled with their corresponding
res/req and cannot be treated independently as
normal streams. Add a special exception for this
in the pipeline cleanup.

Fixes: https://github.com/nodejs/node/issues/32184

PR-URL: https://github.com/nodejs/node/pull/32197
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2020-03-11 15:03:31 -04:00
Matteo Collina
d7fe5548ef stream: do not swallow errors with async iterators and pipeline
Before this patch, pipeline() could swallow errors by pre-emptively
producing a ERR_STREAM_PREMATURE_CLOSE that was not really helpful
to the user.

Co-Authored-By: Robert Nagy <ronagy@icloud.com>

PR-URL: https://github.com/nodejs/node/pull/32051
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2020-03-11 16:22:22 +01:00
Robert Nagy
4d93e105bf stream: don't destroy final readable stream in pipeline
If the last stream in a pipeline is still usable/readable
don't destroy it to allow further composition.

Fixes: https://github.com/nodejs/node/issues/32105

PR-URL: https://github.com/nodejs/node/pull/32110
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2020-03-08 10:54:11 +01:00
Robert Nagy
960be159ac stream: add comments to pipeline implementation
Fixes: https://github.com/nodejs/node/issues/32039

PR-URL: https://github.com/nodejs/node/pull/32042
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2020-03-07 18:17:54 +01:00
Robert Nagy
7cafd5f3a9 stream: fix finished w/ 'close' before 'end'
Emitting 'close' before 'end' on a Readable should
result in a premature close error.

PR-URL: https://github.com/nodejs/node/pull/31545
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
2020-03-03 13:34:24 +01:00
Robert Nagy
b2be348fcc stream: ensure pipeline always destroys streams
There was an edge case where an incorrect assumption was made
in regardos whether eos/finished means that the stream is
actually destroyed or not.

PR-URL: https://github.com/nodejs/node/pull/31940
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2020-02-26 23:48:14 +01:00
Robert Nagy
db28739aed stream: fix broken pipeline error propagation
If the destination was an async function any
error thrown from that function would be swallowed.

PR-URL: https://github.com/nodejs/node/pull/31835
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
2020-02-24 12:20:49 +01:00
Robert Nagy
07915db233 stream: re-use legacy destroyer
PR-URL: https://github.com/nodejs/node/pull/31316
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2020-01-26 17:53:13 +01:00
Robert Nagy
c6bf9539cc stream: simplify pipeline
PR-URL: https://github.com/nodejs/node/pull/31316
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2020-01-26 17:53:10 +01:00
Robert Nagy
7b78ff0428 stream: support passing generator functions into pipeline()
PR-URL: https://github.com/nodejs/node/pull/31223
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
2020-01-17 16:50:07 -08:00
Robert Nagy
c852f7e2ac stream: pipeline should use req.abort() to destroy response
destroy(err) on http response will propagate the error to the
request causing 'error' to be unexpectedly emitted. Furthermore,
response.destroy() unlike request.abort() does not _dump buffered
data.

Fixes a breaking change introduced in 648088289d.

Prefer res.req.abort() over res.destroy() until this situation is
clarified.

Fixes: https://github.com/nodejs/node/issues/31029
Refs: 648088289d

PR-URL: https://github.com/nodejs/node/pull/31054
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2019-12-25 12:50:30 -08:00
Matteo Collina
648088289d stream: make all streams error in a pipeline
This changes makes all stream in a pipeline emit 'error' in
case of an abnormal termination of the pipeline. If the last stream
is currently being async iterated, this change will make the iteration
reject accordingly.

See: https://github.com/nodejs/node/pull/30861
Fixes: https://github.com/nodejs/node/issues/28194

PR-URL: https://github.com/nodejs/node/pull/30869
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
2019-12-14 16:16:34 +01:00
Michaël Zasso
141a6e34ee
lib: enforce use of Array from primordials
PR-URL: https://github.com/nodejs/node/pull/30635
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2019-11-27 19:29:01 +01:00
Ruben Bridgewater
3b044962c4
errors: add more information in case of invalid callbacks
This adds the actual callback that is passed through to the error
message in case an ERR_INVALID_CALLBACK error is thrown.

PR-URL: https://github.com/nodejs/node/pull/27048
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Yongsheng Zhang <zyszys98@gmail.com>
2019-04-04 12:51:03 +02:00
cjihrig
466cda0548 fs: ensure readdir() callback is only called once
This commit ensures that the readdir() callback can only be
called once when the withFileTypes parameter is supplied.

PR-URL: https://github.com/nodejs/node/pull/22793
Fixes: https://github.com/nodejs/node/issues/22778
Reviewed-By: Bryan English <bryan@bryanenglish.com>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
2018-09-13 08:13:17 -04:00
Ruben Bridgewater
32c51f10d3
stream: make the pipeline callback mandatory
Right now when not adding a callback to the pipeline it could cause
an uncaught exception if there is an error. Instead, just make the
callback mandatory as mostly done in all other Node.js callback APIs
so users explicitly have to decide what to do in such situations.

PR-URL: https://github.com/nodejs/node/pull/21054
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2018-06-08 01:24:39 +02:00
Blaine Bublitz
c1012b4402
stream: ensure Stream.pipeline re-throws errors without callback
Fixes an issue where Stream.pipeline wouldn't re-throw errors
on a stream if no callback was specified, thus swallowing
said errors.

Fixes: https://github.com/nodejs/node/issues/20303

PR-URL: https://github.com/nodejs/node/pull/20437
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2018-05-31 11:22:54 +02:00
Ruben Bridgewater
bf46c371eb
stream: lazy load end-of-stream
PR-URL: https://github.com/nodejs/node/pull/20567
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
2018-05-18 15:27:29 +02:00
Mathias Buus
f64bebf205 stream: add pipeline and finished
PR-URL: https://github.com/nodejs/node/pull/19828
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2018-04-16 16:02:12 +02:00