Previously when managing the importModuleDynamically callback of
vm.compileFunction(), we use an ID number as the host defined option
and maintain a per-Environment ID -> CompiledFnEntry map to retain
the top-level referrer function returned by vm.compileFunction() in
order to pass it back to the callback, but it would leak because with
how we used v8::Persistent to maintain this reference, V8 would not
be able to understand the cycle and would just think that the
CompiledFnEntry was supposed to live forever. We made an attempt
to make that reference known to V8 by making the CompiledFnEntry weak
and using a private symbol to make CompiledFnEntry strongly
references the top-level referrer function in
https://github.com/nodejs/node/pull/46785, but that turned out to be
unsound, because the there's no guarantee that the top-level function
must be alive while import() can still be initiated from that
function, since V8 could discard the top-level function and only keep
inner functions alive, so relying on the top-level function to keep
the CompiledFnEntry alive could result in use-after-free which caused
a revert of that fix.
With this patch we use a symbol in the host defined options instead of
a number, because with the stage-3 symbol-as-weakmap-keys proposal
we could directly use that symbol to keep the referrer alive using a
WeakMap. As a bonus this also keeps the other kinds of referrers
alive as long as import() can still be initiated from that
Script/Module, so this also fixes the long-standing crash caused by
vm.Script being GC'ed too early when its importModuleDynamically
callback still needs it.
PR-URL: https://github.com/nodejs/node/pull/48510
Refs: https://github.com/nodejs/node/issues/44211
Refs: https://github.com/nodejs/node/issues/42080
Refs: https://github.com/nodejs/node/issues/47096
Refs: https://github.com/nodejs/node/issues/43205
Refs: https://github.com/nodejs/node/issues/38695
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
PR-URL: https://github.com/nodejs/node/pull/46305
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
This moves the following utils into modules/esm/utils.js:
- Code related to default conditions
- The callbackMap (which is now created in the module instead of
hanging off the module_wrap binding, since the C++ land
does not need it).
- Per-isolate module callbacks
These are self-contained code that can be included into the
built-in snapshot.
PR-URL: https://github.com/nodejs/node/pull/45849
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
The helpers are actually shared by the two loaders, so move them
under modules/ directly.
PR-URL: https://github.com/nodejs/node/pull/45849
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
It turns out that even with startup snapshots, there is a non-trivial
overhead for loading internal modules. This patch makes the loading
of the non-essential modules lazy again.
Caveat: we have to make some of the globals lazily-loaded too,
so the WPT runner is updated to test what the state of the global
scope is after the globals are accessed (and replaced with the
loaded value).
PR-URL: https://github.com/nodejs/node/pull/45659
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Tierney Cyren <hello@bnb.im>
The term "native module" dates back to some of the oldest code
in the code base. Within the context of Node.js core it usually
refers to modules that are native to Node.js (e.g. fs, http),
but it can cause confusion for people who don't work on this
part of the code base, as "native module" can also refer to
native addons - which is even the case in some of the API
docs and error messages.
This patch tries to make the usage of these terms more consistent.
Now within the context of Node.js core:
- JavaScript scripts that are built-in to Node.js are now referred
to as "built-in(s)". If they are available as modules,
they can also be referred to as "built-in module(s)".
- Dynamically-linked shared objects that are loaded into
the Node.js processes are referred to as "addons".
We will try to avoid using the term "native modules" because it could
be ambiguous.
Changes in this patch:
File names:
- node_native_module.h -> node_builtins.h,
- node_native_module.cc -> node_builtins.cc
C++ binding names:
- `native_module` -> `builtins`
`node::Environment`:
- `native_modules_without_cache` -> `builtins_without_cache`
- `native_modules_with_cache` -> `builtins_with_cache`
- `native_modules_in_snapshot` -> `builtins_in_cache`
- `native_module_require` -> `builtin_module_require`
`node::EnvSerializeInfo`:
- `native_modules` -> `builtins
`node::native_module::NativeModuleLoader`:
- `native_module` namespace -> `builtins` namespace
- `NativeModuleLoader` -> `BuiltinLoader`
- `NativeModuleRecordMap` -> `BuiltinSourceMap`
- `NativeModuleCacheMap` -> `BuiltinCodeCacheMap`
- `ModuleIds` -> `BuiltinIds`
- `ModuleCategories` -> `BuiltinCategories`
- `LoadBuiltinModuleSource` -> `LoadBuiltinSource`
`loader.js`:
- `NativeModule` -> `BuiltinModule` (the `NativeModule` name used in
`process.moduleLoadList` is kept for compatibility)
And other clarifications in the documentation and comments.
PR-URL: https://github.com/nodejs/node/pull/44135
Fixes: https://github.com/nodejs/node/issues/44036
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/43612
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/43164
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
PR-URL: https://github.com/nodejs/node/pull/40785
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Remove V8 flag for import assertions, enabling support for the syntax;
require the import assertion syntax for imports of JSON.
Support import assertions in user loaders.
Use both resolved module URL and import assertion type as the key for
caching modules.
Co-authored-by: Geoffrey Booth <webadmin@geoffreybooth.com>
PR-URL: https://github.com/nodejs/node/pull/40250
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
doc: update ESM hook examples
esm: fix unsafe primordial
doc: fix ESM example linting
esm: allow source of type ArrayBuffer
doc: update ESM hook changelog to include resolve format
esm: allow all ArrayBuffers and TypedArrays for load hook source
doc: tidy code & API docs
doc: convert ESM source table header from Title Case to Sentence case
doc: add detailed explanation for getPackageType
esm: add caveat that ESMLoader::import() must NOT be renamed
esm: tidy code declaration of getFormat protocolHandlers
doc: correct ESM doc link (bad conflict resolution)
doc: update ESM hook limitation for CJS
esm: tweak preload description
doc: update ESM getPackageType() example explanation
PR-URL: https://github.com/nodejs/node/pull/37468
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Geoffrey Booth <webmaster@geoffreybooth.com>
Fixes: https://github.com/nodejs/node/issues/36098
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Guy Bedford <guybedford@gmail.com>
Co-authored-by: Darshan Sen <raisinten@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/37246
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
It is guaranteed that V8 throws a syntax error when `import` or `export`
is used outside of ESM.
Fixes: https://github.com/nodejs/node/issues/35687
PR-URL: https://github.com/nodejs/node/pull/35691
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Store all primordials as properties of the primordials object.
Static functions are prefixed by the constructor's name and prototype
methods are prefixed by the constructor's name followed by "Prototype".
For example: primordials.Object.keys becomes primordials.ObjectKeys.
PR-URL: https://github.com/nodejs/node/pull/30610
Refs: https://github.com/nodejs/node/issues/29766
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/29564
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
