Files in `node_modules` are not authored by the user directly and the
original sources are less relevant to the user.
Skipping source maps in `node_modules` improves the general
performance. Add `module.setSourceMapsSupport(enabled, options)` to
skip source maps in `node_modules` if it is needed. This moves
all source maps related API to `node:module` and this a step to
promote the source maps API to stable.
PR-URL: https://github.com/nodejs/node/pull/56639
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.
PR-URL: https://github.com/nodejs/node/pull/55520
Fixes: https://github.com/nodejs/node/issues/55516
Refs: https://github.com/nodejs/node/issues/52697
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
When a ESM module cannot be loaded by require due to the presence
of TLA, its module status would be stopped at kInstantiated. In
this case, when it's imported again, we should allow it to be
evaluated asynchronously, as it's also a common pattern for users
to retry with dynamic import when require fails.
PR-URL: https://github.com/nodejs/node/pull/55502
Fixes: https://github.com/nodejs/node/issues/55500
Refs: https://github.com/nodejs/node/issues/52697
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.
There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.
PR-URL: https://github.com/nodejs/node/pull/55085
Refs: https://github.com/nodejs/node/issues/52697
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: LiviaMedeiros <livia@cirno.name>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
This lays the foundation for supporting synchronous hooks proposed
in https://github.com/nodejs/loaders/pull/198 for ESM.
- Corrects and adds several JSDoc comments for internal functions
of the ESM loader, as well as explaining how require() for
import CJS work in the special resolve/load paths. This doesn't
consolidate it with import in require(esm) yet due to caching
differences, which is left as a TODO.
- The moduleProvider passed into ModuleJob is replaced as
moduleOrModulePromise, we call the translators directly in the
ESM loader and verify it right after loading for clarity.
- Reuse a few refactored out helpers for require(esm) in
getModuleJobForRequire().
PR-URL: https://github.com/nodejs/node/pull/54769
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
So that if there are circular dependencies in the synchronous
module graph, they could be resolved using the cached jobs.
In case linking fails and the error gets caught, reset the
cache right after linking. If it succeeds, the caller will
cache it again. Otherwise the error bubbles up to users,
and since we unset the cache for the unlinkable module
the next attempt would still fail.
PR-URL: https://github.com/nodejs/node/pull/52868
Fixes: https://github.com/nodejs/node/issues/52864
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Avoid repetitively calling into JS callback from C++ in
`ModuleWrap::Link`. This removes the convoluted callback style of the
internal `ModuleWrap` link step.
PR-URL: https://github.com/nodejs/node/pull/52058
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Previously there is an edge case where submodules loaded by require()
may not be loaded by import() again from different intermediate
edges in the graph. This patch fixes that, added tests, and added
debug logs.
Drive-by: make loader a private field so it doesn't show up in logs.
PR-URL: https://github.com/nodejs/node/pull/52487
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
This patch adds `require()` support for synchronous ESM graphs under
the flag `--experimental-require-module`
This is based on the the following design aspect of ESM:
- The resolution can be synchronous (up to the host)
- The evaluation of a synchronous graph (without top-level await) is
also synchronous, and, by the time the module graph is instantiated
(before evaluation starts), this is is already known.
If `--experimental-require-module` is enabled, and the ECMAScript
module being loaded by `require()` meets the following requirements:
- Explicitly marked as an ES module with a `"type": "module"` field in
the closest package.json or a `.mjs` extension.
- Fully synchronous (contains no top-level `await`).
`require()` will load the requested module as an ES Module, and return
the module name space object. In this case it is similar to dynamic
`import()` but is run synchronously and returns the name space object
directly.
```mjs
// point.mjs
export function distance(a, b) {
return (b.x - a.x) ** 2 + (b.y - a.y) ** 2;
}
class Point {
constructor(x, y) { this.x = x; this.y = y; }
}
export default Point;
```
```cjs
const required = require('./point.mjs');
// [Module: null prototype] {
// default: [class Point],
// distance: [Function: distance]
// }
console.log(required);
(async () => {
const imported = await import('./point.mjs');
console.log(imported === required); // true
})();
```
If the module being `require()`'d contains top-level `await`, or the
module graph it `import`s contains top-level `await`,
[`ERR_REQUIRE_ASYNC_MODULE`][] will be thrown. In this case, users
should load the asynchronous module using `import()`.
If `--experimental-print-required-tla` is enabled, instead of throwing
`ERR_REQUIRE_ASYNC_MODULE` before evaluation, Node.js will evaluate the
module, try to locate the top-level awaits, and print their location to
help users fix them.
PR-URL: https://github.com/nodejs/node/pull/51977
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
When the entry point is a module and the graph it imports still
contains unsettled top-level await when the Node.js instance
finishes the event loop, search from the entry point module
for unsettled top-level await and print their location.
To avoid unnecessary overhead, we register a promise that only
gets settled when the entry point graph evaluation returns
from await, and only search the module graph if it's still
unsettled by the time the instance is exiting.
This patch only handles this for entry point modules. Other kinds of
modules are more complicated so will be left for the future.
Drive-by: update the terminology "unfinished promise" to the
more correct one "unsettled promise" in the codebase.
PR-URL: https://github.com/nodejs/node/pull/51999
Fixes: https://github.com/nodejs/node/issues/42868
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Previously we only had an internal assertion to ensure certain
code is executed before any user-provided CJS is run. This patch
adds another assertion for ESM.
Note that this internal state is not updated during source text
module execution via vm because to run any code via vm, some
user JS code must have already been executed anyway.
In addition this patch moves the states into internal/modules/helpers
to avoid circular dependencies. Also moves toggling the states to
true *right before* user code execution instead of after in case
we are half-way in the execution when internals try to check them.
PR-URL: https://github.com/nodejs/node/pull/51748
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
PR-URL: https://github.com/nodejs/node/pull/50322
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Co-authored-by: Daniel Lemire <daniel@lemire.me>
PR-URL: https://github.com/nodejs/node/pull/50322
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
The old import assertions proposal has been
renamed to "import attributes" with the follwing major changes:
1. The keyword is now `with` instead of `assert`.
2. Unknown assertions cause an error rather than being ignored,
This commit updates the documentation to encourage folks to use the new
syntax, and add aliases for module customization hooks.
PR-URL: https://github.com/nodejs/node/pull/50140
Fixes: https://github.com/nodejs/node/issues/50134
Refs: 159c82c5e6
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Major functional changes:
- Allow `import()` to work within loaders that require other loaders,
- Unflag the use of `Module.register`.
A new interface `Customizations` has been created in order to unify
`ModuleLoader` (previously `DefaultModuleLoader`), `Hooks` and
`CustomizedModuleLoader` all of which now implement it:
```ts
interface LoadResult {
format: ModuleFormat;
source: ModuleSource;
}
interface ResolveResult {
format: string;
url: URL['href'];
}
interface Customizations {
allowImportMetaResolve: boolean;
load(url: string, context: object): Promise<LoadResult>
resolve(
originalSpecifier:
string, parentURL: string,
importAssertions: Record<string, string>
): Promise<ResolveResult>
resolveSync(
originalSpecifier:
string, parentURL: string,
importAssertions: Record<string, string>
) ResolveResult;
register(specifier: string, parentUrl: string): any;
forceLoadHooks(): void;
importMetaInitialize(meta, context, loader): void;
}
```
The `ModuleLoader` class now has `setCustomizations` which takes an
object of this shape and delegates its responsibilities to this object
if present.
Note that two properties `allowImportMetaResolve` and `resolveSync`
exist now as a mechanism for `import.meta.resolve` – since `Hooks`
does not implement `resolveSync` other loaders cannot use
`import.meta.resolve`; `allowImportMetaResolve` is a way of checking
for that case instead of invoking `resolveSync` and erroring.
Fixes https://github.com/nodejs/node/issues/48515
Closes https://github.com/nodejs/node/pull/48439
PR-URL: https://github.com/nodejs/node/pull/48559
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
fixup: add support for `Object.create(null)`
fixup: extend to any 1-argument Object.create call
fixup: add tests
PR-URL: https://github.com/nodejs/node/pull/46083
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
In a previous commit, the loader implementation was modified to be
protected against most prototype pollution, but was kept vulnerable to
`Array.prototype` pollution. This commit fixes that, the tradeoff is
that it modifies the `ESMLoader.prototype.import` return type from an
`Array` to an array-like object.
Refs: https://github.com/nodejs/node/pull/45044
PR-URL: https://github.com/nodejs/node/pull/45175
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Fixes: https://github.com/nodejs/node/issues/45035
PR-URL: https://github.com/nodejs/node/pull/45044
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/43164
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
PR-URL: https://github.com/nodejs/node/pull/43130
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Remove V8 flag for import assertions, enabling support for the syntax;
require the import assertion syntax for imports of JSON.
Support import assertions in user loaders.
Use both resolved module URL and import assertion type as the key for
caching modules.
Co-authored-by: Geoffrey Booth <webadmin@geoffreybooth.com>
PR-URL: https://github.com/nodejs/node/pull/40250
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
doc: update ESM hook examples
esm: fix unsafe primordial
doc: fix ESM example linting
esm: allow source of type ArrayBuffer
doc: update ESM hook changelog to include resolve format
esm: allow all ArrayBuffers and TypedArrays for load hook source
doc: tidy code & API docs
doc: convert ESM source table header from Title Case to Sentence case
doc: add detailed explanation for getPackageType
esm: add caveat that ESMLoader::import() must NOT be renamed
esm: tidy code declaration of getFormat protocolHandlers
doc: correct ESM doc link (bad conflict resolution)
doc: update ESM hook limitation for CJS
esm: tweak preload description
doc: update ESM getPackageType() example explanation
PR-URL: https://github.com/nodejs/node/pull/37468
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Geoffrey Booth <webmaster@geoffreybooth.com>
Logic in module_job.js assumes detailed stack trace from node_errors.cc
which is not populated when --enable-source-maps is set.
Fixes#38790
PR-URL: https://github.com/nodejs/node/pull/39017
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
PR-URL: https://github.com/nodejs/node/pull/37773
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com>
Reviewed-By: Pooja D P <Pooja.D.P@ibm.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Zijian Liu <lxxyxzj@gmail.com>
The node process crashes when trying to parse a multiline import
statement for named exports of a CommonJS module:
TypeError: Cannot read property '0' of null
at ModuleJob._instantiate (internal/modules/esm/module_job.js:112:77)
at async ModuleJob.run (internal/modules/esm/module_job.js:137:5)
at async Loader.import (internal/modules/esm/loader.js:165:24)
at async rejects.name (file:///***/node/test/es-module/test-esm-cjs-named-error.mjs:56:3)
at async waitForActual (assert.js:721:5)
at async rejects (assert.js:830:25),
The reason is that the regexp that is currently used to decorate the
original error fails for multi line import statements.
Unfortunately the undecorated error stack only contains the single line
which causes the import to fail:
file:///***/node/test/fixtures/es-modules/package-cjs-named-error/multi-line.mjs:2
comeOn,
^^^^^^
SyntaxError: The requested module './fail.cjs' does not provide an export named 'comeOn'
at ModuleJob._instantiate (internal/modules/esm/module_job.js:98:21)
at async ModuleJob.run (internal/modules/esm/module_job.js:141:5)
at async Loader.import (internal/modules/esm/loader.js:165:24)
at async rejects.name (file:///***/node/test/es-module/test-esm-cjs-named-error.mjs:56:3)
at async waitForActual (assert.js:721:5)
at async rejects (assert.js:830:25)
Hence, for multiline import statements we cannot create an equivalent
piece of code that uses default import followed by an object
destructuring assignment.
In any case the node process should definitely not crash. So until we
have a more sophisticated way of extracting the entire problematic
multiline import statement, show the code example only for single-line
imports where the current regexp approach works well.
Refs: https://github.com/nodejs/node/issues/35259
PR-URL: https://github.com/nodejs/node/pull/35275
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
When importing specific names from a CJS module, and renaming them using
`as`, the example fix in the error message erroneously contains the
keyword `as` in the destructuring variable declaration.
Example of this issue:
import { parse as acornParse } from "acorn";
^^^^^
SyntaxError: The requested module 'acorn' is expected to be of type CommonJS, which does not support named exports. CommonJS modules can be imported by importing the default export.
For example:
import pkg from 'acorn';
const { parse as acornParse } = pkg;
PR-URL: https://github.com/nodejs/node/pull/33882
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
We do not support importing named exports from a CJS module.
This change decorates the error message for missing named exports in
the case where the module being imported is expected to be CJS by the
ESM loader.
Signed-off-by: Myles Borins <myles.borins@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/33256
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Store all primordials as properties of the primordials object.
Static functions are prefixed by the constructor's name and prototype
methods are prefixed by the constructor's name followed by "Prototype".
For example: primordials.Object.keys becomes primordials.ObjectKeys.
PR-URL: https://github.com/nodejs/node/pull/30610
Refs: https://github.com/nodejs/node/issues/29766
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
