Commit Graph

22 Commits

Author SHA1 Message Date
Robert Nagy
98b6b2d840 stream: complete pipeline with stdio
stdio (stderr & stdout) should for compatibility
reasons not be closed/end():ed. However, this
causes pipeline with a stdio destination to
never finish. This commit fixes this issue at
a performance cost.

Refs: https://github.com/nodejs/node/issues/7606

Fixes: https://github.com/nodejs/node/issues/32363

PR-URL: https://github.com/nodejs/node/pull/32373
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2020-04-06 21:48:25 +02:00
Robert Nagy
05f1df5200 stream: fix pipeline with dest in objectMode
pipeline did not support destination with generator
that does not return strings or buffers.

PR-URL: https://github.com/nodejs/node/pull/32414
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2020-03-25 14:46:10 +01:00
Robert Nagy
2bfb3403ed
stream: avoid destroying writable source
User might still want to be able to use the writable side
of src. This is in the case where e.g. the Duplex input
is not directly connected to its output. Such a case could
happen when the Duplex is reading from a socket and then echos
the data back on the same socket.

PR-URL: https://github.com/nodejs/node/pull/32198
Refs: 4d93e105bf (commitcomment-37751035)
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2020-03-11 20:57:42 -04:00
Robert Nagy
f8f20892e9
stream: avoid destroying http1 objects
http1 objects are coupled with their corresponding
res/req and cannot be treated independently as
normal streams. Add a special exception for this
in the pipeline cleanup.

Fixes: https://github.com/nodejs/node/issues/32184

PR-URL: https://github.com/nodejs/node/pull/32197
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2020-03-11 15:03:31 -04:00
Matteo Collina
d7fe5548ef stream: do not swallow errors with async iterators and pipeline
Before this patch, pipeline() could swallow errors by pre-emptively
producing a ERR_STREAM_PREMATURE_CLOSE that was not really helpful
to the user.

Co-Authored-By: Robert Nagy <ronagy@icloud.com>

PR-URL: https://github.com/nodejs/node/pull/32051
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2020-03-11 16:22:22 +01:00
Robert Nagy
4d93e105bf stream: don't destroy final readable stream in pipeline
If the last stream in a pipeline is still usable/readable
don't destroy it to allow further composition.

Fixes: https://github.com/nodejs/node/issues/32105

PR-URL: https://github.com/nodejs/node/pull/32110
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2020-03-08 10:54:11 +01:00
Robert Nagy
960be159ac stream: add comments to pipeline implementation
Fixes: https://github.com/nodejs/node/issues/32039

PR-URL: https://github.com/nodejs/node/pull/32042
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2020-03-07 18:17:54 +01:00
Robert Nagy
7cafd5f3a9 stream: fix finished w/ 'close' before 'end'
Emitting 'close' before 'end' on a Readable should
result in a premature close error.

PR-URL: https://github.com/nodejs/node/pull/31545
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
2020-03-03 13:34:24 +01:00
Robert Nagy
b2be348fcc stream: ensure pipeline always destroys streams
There was an edge case where an incorrect assumption was made
in regardos whether eos/finished means that the stream is
actually destroyed or not.

PR-URL: https://github.com/nodejs/node/pull/31940
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2020-02-26 23:48:14 +01:00
Robert Nagy
db28739aed stream: fix broken pipeline error propagation
If the destination was an async function any
error thrown from that function would be swallowed.

PR-URL: https://github.com/nodejs/node/pull/31835
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
2020-02-24 12:20:49 +01:00
Robert Nagy
07915db233 stream: re-use legacy destroyer
PR-URL: https://github.com/nodejs/node/pull/31316
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2020-01-26 17:53:13 +01:00
Robert Nagy
c6bf9539cc stream: simplify pipeline
PR-URL: https://github.com/nodejs/node/pull/31316
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2020-01-26 17:53:10 +01:00
Robert Nagy
7b78ff0428 stream: support passing generator functions into pipeline()
PR-URL: https://github.com/nodejs/node/pull/31223
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
2020-01-17 16:50:07 -08:00
Robert Nagy
c852f7e2ac stream: pipeline should use req.abort() to destroy response
destroy(err) on http response will propagate the error to the
request causing 'error' to be unexpectedly emitted. Furthermore,
response.destroy() unlike request.abort() does not _dump buffered
data.

Fixes a breaking change introduced in 648088289d.

Prefer res.req.abort() over res.destroy() until this situation is
clarified.

Fixes: https://github.com/nodejs/node/issues/31029
Refs: 648088289d

PR-URL: https://github.com/nodejs/node/pull/31054
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2019-12-25 12:50:30 -08:00
Matteo Collina
648088289d stream: make all streams error in a pipeline
This changes makes all stream in a pipeline emit 'error' in
case of an abnormal termination of the pipeline. If the last stream
is currently being async iterated, this change will make the iteration
reject accordingly.

See: https://github.com/nodejs/node/pull/30861
Fixes: https://github.com/nodejs/node/issues/28194

PR-URL: https://github.com/nodejs/node/pull/30869
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
2019-12-14 16:16:34 +01:00
Michaël Zasso
141a6e34ee
lib: enforce use of Array from primordials
PR-URL: https://github.com/nodejs/node/pull/30635
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
2019-11-27 19:29:01 +01:00
Ruben Bridgewater
3b044962c4
errors: add more information in case of invalid callbacks
This adds the actual callback that is passed through to the error
message in case an ERR_INVALID_CALLBACK error is thrown.

PR-URL: https://github.com/nodejs/node/pull/27048
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Yongsheng Zhang <zyszys98@gmail.com>
2019-04-04 12:51:03 +02:00
cjihrig
466cda0548 fs: ensure readdir() callback is only called once
This commit ensures that the readdir() callback can only be
called once when the withFileTypes parameter is supplied.

PR-URL: https://github.com/nodejs/node/pull/22793
Fixes: https://github.com/nodejs/node/issues/22778
Reviewed-By: Bryan English <bryan@bryanenglish.com>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
2018-09-13 08:13:17 -04:00
Ruben Bridgewater
32c51f10d3
stream: make the pipeline callback mandatory
Right now when not adding a callback to the pipeline it could cause
an uncaught exception if there is an error. Instead, just make the
callback mandatory as mostly done in all other Node.js callback APIs
so users explicitly have to decide what to do in such situations.

PR-URL: https://github.com/nodejs/node/pull/21054
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2018-06-08 01:24:39 +02:00
Blaine Bublitz
c1012b4402
stream: ensure Stream.pipeline re-throws errors without callback
Fixes an issue where Stream.pipeline wouldn't re-throw errors
on a stream if no callback was specified, thus swallowing
said errors.

Fixes: https://github.com/nodejs/node/issues/20303

PR-URL: https://github.com/nodejs/node/pull/20437
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2018-05-31 11:22:54 +02:00
Ruben Bridgewater
bf46c371eb
stream: lazy load end-of-stream
PR-URL: https://github.com/nodejs/node/pull/20567
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
2018-05-18 15:27:29 +02:00
Mathias Buus
f64bebf205 stream: add pipeline and finished
PR-URL: https://github.com/nodejs/node/pull/19828
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2018-04-16 16:02:12 +02:00