Divide builtins into two lists depending on whether
they are loaded before pre-execution or at run time,
and give clearer suggestions about how to deal with them
based on the category they are in.
This helps preventing regressions like the one reported
in https://github.com/nodejs/node/issues/45662.
PR-URL: https://github.com/nodejs/node/pull/50708
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Co-authored-by: Daniel Lemire <daniel@lemire.me>
PR-URL: https://github.com/nodejs/node/pull/50322
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Previously to throw errors from C++ land, sync versions of the fs
were created by copying C++ code from the original implementation
and moving JS code to a separate file. This can lead to several
problems:
1. By moving code to a new file for the sake of moving, it would
be harder to use git blame to trace changes and harder to backport
changes to older branches.
2. Scattering the async and sync versions of fs methods in
different files makes it harder to keep them in sync and
share code in the prologues and epilogues.
3. Having two copies of code doing almost the same thing results
in duplication and can be prone to out-of-sync problems when the
prologue and epilogue get updated.
4. There is a minor cost to startup in adding an additional file.
This can add up even with the help of snapshots.
This patch moves the JS code back to lib/fs.js to stop 1, 2 & 4
and introduces C++ helpers SyncCallAndThrowIf() and
SyncCallAndThrowOnError() so that the original implementations
can be easily tweaked to allow throwing from C++ and stop 3.
PR-URL: https://github.com/nodejs/node/pull/49913
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
PR-URL: https://github.com/nodejs/node/pull/48477
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Move the bindings used by TextEncoder to a new binding for
more self-contained code.
PR-URL: https://github.com/nodejs/node/pull/46658
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Move the logic for handling --test-reporter out of the
general module loader and into the test_runner subsystem.
PR-URL: https://github.com/nodejs/node/pull/45923
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Preload essential modules and lazy-load non-essential ones.
After this patch, all modules listed by running this snippet:
```
const list = process.moduleLoadList.join('\n');
require('fs').writeSync(1, list, 'utf-8');
```
(which is roughly the same list as the one in
test-bootstrap-module.js for the main thread)
are loaded from the snapshot so no additional compilation cost
is incurred.
PR-URL: https://github.com/nodejs/node/pull/45849
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This patch makes the top-level access to runtime states in the
CJS loader lazy, and move the side-effects into a
initializeCJS() function that gets called during pre-execution.
As a result the CJS loader can be included into the built-in
snapshot.
PR-URL: https://github.com/nodejs/node/pull/45849
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
This moves the following utils into modules/esm/utils.js:
- Code related to default conditions
- The callbackMap (which is now created in the module instead of
hanging off the module_wrap binding, since the C++ land
does not need it).
- Per-isolate module callbacks
These are self-contained code that can be included into the
built-in snapshot.
PR-URL: https://github.com/nodejs/node/pull/45849
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
So that the file can be snapshotted.
PR-URL: https://github.com/nodejs/node/pull/45849
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
The helpers are actually shared by the two loaders, so move them
under modules/ directly.
PR-URL: https://github.com/nodejs/node/pull/45849
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/45808
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
It turns out that even with startup snapshots, there is a non-trivial
overhead for loading internal modules. This patch makes the loading
of the non-essential modules lazy again.
Caveat: we have to make some of the globals lazily-loaded too,
so the WPT runner is updated to test what the state of the global
scope is after the globals are accessed (and replaced with the
loaded value).
PR-URL: https://github.com/nodejs/node/pull/45659
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Tierney Cyren <hello@bnb.im>
V8 already parses the source map magic comments. Currently, only scripts
and functions expose the parsed source map URLs. It is unnecessary to
parse the source map magic comments again when the parsed information is
available.
PR-URL: https://github.com/nodejs/node/pull/44798
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Co-authored-by: Rich Trott <rtrott@gmail.com>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/21128
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Enables `--experimental-global-webcrypto` by default, and ensures that
the classic `node:crypto` core module is still available in `--eval` or
`--print` contexts.
PR-URL: https://github.com/nodejs/node/pull/42083
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
This patch refactors the DNS default resolver code to make it
easier to be included in a snapshot:
- The code specific for the callback-based DNS resolver are not
in a separate module to make the dependency clearer (it's not
actually needed if the user only ever loads `dns/promises`)
- The common part of the callback-based resolver and the promise-
based resolver is now ResolverBase. The other two user-facing
resolvers are now subclasses of ResolverBase. The default
Resolver is constructed with just ResolverBase. This would
be fine as the default resolver is never actually exposed
to the user-land and it has been working using duck-typing anyway.
- Move the construction of Resolver subclasses into a common
method `createResolverClass()` to reduce code duplication.
The two subclasses now also share the same base constructor.
This would make it possible for them to also share code
for snapshot support later.
- `--dns-result-order` is now queried and refreshed during
pre-execution. To avoid loading the cares_wrap binding unnecessarily
the loading of the binding is also made lazy.
PR-URL: https://github.com/nodejs/node/pull/44541
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
The term "native module" dates back to some of the oldest code
in the code base. Within the context of Node.js core it usually
refers to modules that are native to Node.js (e.g. fs, http),
but it can cause confusion for people who don't work on this
part of the code base, as "native module" can also refer to
native addons - which is even the case in some of the API
docs and error messages.
This patch tries to make the usage of these terms more consistent.
Now within the context of Node.js core:
- JavaScript scripts that are built-in to Node.js are now referred
to as "built-in(s)". If they are available as modules,
they can also be referred to as "built-in module(s)".
- Dynamically-linked shared objects that are loaded into
the Node.js processes are referred to as "addons".
We will try to avoid using the term "native modules" because it could
be ambiguous.
Changes in this patch:
File names:
- node_native_module.h -> node_builtins.h,
- node_native_module.cc -> node_builtins.cc
C++ binding names:
- `native_module` -> `builtins`
`node::Environment`:
- `native_modules_without_cache` -> `builtins_without_cache`
- `native_modules_with_cache` -> `builtins_with_cache`
- `native_modules_in_snapshot` -> `builtins_in_cache`
- `native_module_require` -> `builtin_module_require`
`node::EnvSerializeInfo`:
- `native_modules` -> `builtins
`node::native_module::NativeModuleLoader`:
- `native_module` namespace -> `builtins` namespace
- `NativeModuleLoader` -> `BuiltinLoader`
- `NativeModuleRecordMap` -> `BuiltinSourceMap`
- `NativeModuleCacheMap` -> `BuiltinCodeCacheMap`
- `ModuleIds` -> `BuiltinIds`
- `ModuleCategories` -> `BuiltinCategories`
- `LoadBuiltinModuleSource` -> `LoadBuiltinSource`
`loader.js`:
- `NativeModule` -> `BuiltinModule` (the `NativeModule` name used in
`process.moduleLoadList` is kept for compatibility)
And other clarifications in the documentation and comments.
PR-URL: https://github.com/nodejs/node/pull/44135
Fixes: https://github.com/nodejs/node/issues/44036
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
This patch:
- Make NativeModuleLoader::LookupAndCompile() detect parameters based
on module IDs. This allows us to compile more builtins when
generating the embedded bootstrap, including
- internal/per_context/*
- internal/bootstrap/*
- internal/main/*
- Move pre_execution.js to lib/internal/process as it needs to be
compiled as a regular built-in module, unlike other scripts
in lib/internal/bootstrap
- Move markBootstrapComplete() to the performance binding instead of
making it a function-wrapper-based global to reduce number of
special cases.
PR-URL: https://github.com/nodejs/node/pull/44018
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
PR-URL: https://github.com/nodejs/node/pull/43967
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
There are no clear indicators anyone is using the dtrace USDT probes.
ETW support is very intertwined with the dtrace infrastructure. It's not
clear if anyone uses ETW so to keep things simple it too is removed.
Fixes: https://github.com/nodejs/node/issues/43649
PR-URL: https://github.com/nodejs/node/pull/43652
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
PR-URL: https://github.com/nodejs/node/pull/43164
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
To lazy load the run time options, the following properties
are updated from value properties to accessor properties
whose getter would turn them back to a value properties
upon the initial access.
- crypto.constants.defaultCipherList
- crypto.pseudoRandomBytes
- crypto.prng
- crypto.rng
PR-URL: https://github.com/nodejs/node/pull/42203
Refs: https://github.com/nodejs/node/issues/37476
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Jacob Smith <3012099+JakobJingleheimer@users.noreply.github.com>
Co-authored-by: James M Snell <jasnell@gmail.com>
Co-authored-by: Jordan Harband <ljharb@gmail.com>
Co-authored-by: James Sumners <james@sumners.email>
PR-URL: https://github.com/nodejs/node/pull/36328
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Implement the map method on readable stream. This starts the alignment
with the tc39-iterator-helpers proposal and adds a `.map` method to
every Node.js readable stream.
Co-Authored-By: Robert Nagy <ronag@icloud.com>
PR-URL: https://github.com/nodejs/node/pull/40815
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Remove V8 flag for import assertions, enabling support for the syntax;
require the import assertion syntax for imports of JSON.
Support import assertions in user loaders.
Use both resolved module URL and import assertion type as the key for
caching modules.
Co-authored-by: Geoffrey Booth <webadmin@geoffreybooth.com>
PR-URL: https://github.com/nodejs/node/pull/40250
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
PR-URL: https://github.com/nodejs/node/pull/39283
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Reviewed-By: Vladimir de Turckheim <vlad2t@hotmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
doc: update ESM hook examples
esm: fix unsafe primordial
doc: fix ESM example linting
esm: allow source of type ArrayBuffer
doc: update ESM hook changelog to include resolve format
esm: allow all ArrayBuffers and TypedArrays for load hook source
doc: tidy code & API docs
doc: convert ESM source table header from Title Case to Sentence case
doc: add detailed explanation for getPackageType
esm: add caveat that ESMLoader::import() must NOT be renamed
esm: tidy code declaration of getFormat protocolHandlers
doc: correct ESM doc link (bad conflict resolution)
doc: update ESM hook limitation for CJS
esm: tweak preload description
doc: update ESM getPackageType() example explanation
PR-URL: https://github.com/nodejs/node/pull/37468
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Geoffrey Booth <webmaster@geoffreybooth.com>
