Fixes: https://github.com/nodejs/node/issues/45035
PR-URL: https://github.com/nodejs/node/pull/45044
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
The term "native module" dates back to some of the oldest code
in the code base. Within the context of Node.js core it usually
refers to modules that are native to Node.js (e.g. fs, http),
but it can cause confusion for people who don't work on this
part of the code base, as "native module" can also refer to
native addons - which is even the case in some of the API
docs and error messages.
This patch tries to make the usage of these terms more consistent.
Now within the context of Node.js core:
- JavaScript scripts that are built-in to Node.js are now referred
to as "built-in(s)". If they are available as modules,
they can also be referred to as "built-in module(s)".
- Dynamically-linked shared objects that are loaded into
the Node.js processes are referred to as "addons".
We will try to avoid using the term "native modules" because it could
be ambiguous.
Changes in this patch:
File names:
- node_native_module.h -> node_builtins.h,
- node_native_module.cc -> node_builtins.cc
C++ binding names:
- `native_module` -> `builtins`
`node::Environment`:
- `native_modules_without_cache` -> `builtins_without_cache`
- `native_modules_with_cache` -> `builtins_with_cache`
- `native_modules_in_snapshot` -> `builtins_in_cache`
- `native_module_require` -> `builtin_module_require`
`node::EnvSerializeInfo`:
- `native_modules` -> `builtins
`node::native_module::NativeModuleLoader`:
- `native_module` namespace -> `builtins` namespace
- `NativeModuleLoader` -> `BuiltinLoader`
- `NativeModuleRecordMap` -> `BuiltinSourceMap`
- `NativeModuleCacheMap` -> `BuiltinCodeCacheMap`
- `ModuleIds` -> `BuiltinIds`
- `ModuleCategories` -> `BuiltinCategories`
- `LoadBuiltinModuleSource` -> `LoadBuiltinSource`
`loader.js`:
- `NativeModule` -> `BuiltinModule` (the `NativeModule` name used in
`process.moduleLoadList` is kept for compatibility)
And other clarifications in the documentation and comments.
PR-URL: https://github.com/nodejs/node/pull/44135
Fixes: https://github.com/nodejs/node/issues/44036
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
This patch:
- Make NativeModuleLoader::LookupAndCompile() detect parameters based
on module IDs. This allows us to compile more builtins when
generating the embedded bootstrap, including
- internal/per_context/*
- internal/bootstrap/*
- internal/main/*
- Move pre_execution.js to lib/internal/process as it needs to be
compiled as a regular built-in module, unlike other scripts
in lib/internal/bootstrap
- Move markBootstrapComplete() to the performance binding instead of
making it a function-wrapper-based global to reduce number of
special cases.
PR-URL: https://github.com/nodejs/node/pull/44018
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
This commit makes it possible to add new core modules that can
only be require()'ed and imported when the 'node:' scheme is
used. The 'test' module is the first such module.
These 'node:'-only modules are not included in the list returned
by module.builtinModules.
PR-URL: https://github.com/nodejs/node/pull/42325
Refs: https://github.com/nodejs/node/issues/40954
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
These changes are in preparation for enabling a JSDoc lint rule.
PR-URL: https://github.com/nodejs/node/pull/41109
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/38042
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Vladimir de Turckheim <vlad2t@hotmail.com>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
The internal HTTP Parser constructor and methods list are available
via `require('_http_parser')`, there's no need for the ecosystem to
access via `process.binding()`
Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/37813
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
All of the functionality is exposed via public APIs, there are no
APIs that are of use to users directly.
Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/37790
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
The constructor was never meant to be exposed to user code.
Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/37800
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
The only useful functions are already exposed via public APIs.
There's no reason for user code to access the binding.
Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/37789
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/37644
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Pooja D P <Pooja.D.P@ibm.com>
Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/37576
Fixes: https://github.com/nodejs/node/issues/37575
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
We're about to turn on a requirement for dangling commas.
PR-URL: https://github.com/nodejs/node/pull/37088
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Refactor the internal NativeModule class to a JS class and add
more documentation about its properties.
PR-URL: https://github.com/nodejs/node/pull/30856
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Rich Trott <rtrott@gmail.com>
CLI options should not be added through the config binding, instead
they are already available in the JS land via
`require('internal/options')`. Also CLI options in principle should
be processed in pre-execution instead of bootstrap scripts.
PR-URL: https://github.com/nodejs/node/pull/30778
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yongsheng Zhang <zyszys98@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Store all primordials as properties of the primordials object.
Static functions are prefixed by the constructor's name and prototype
methods are prefixed by the constructor's name followed by "Prototype".
For example: primordials.Object.keys becomes primordials.ObjectKeys.
PR-URL: https://github.com/nodejs/node/pull/30610
Refs: https://github.com/nodejs/node/issues/29766
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Due to how bootstrap/loaders.js itself is loaded and invoked,
stacktraces from it are munged and no longer point back to the error
source.
That resulted in the following unhelpful error if an internal module
was missing or misnamed:
```
internal/bootstrap/loaders.js:190
return mod.compile();
^
TypeError: Cannot read property 'compile' of undefined
```
This changes that to at least print the id that was attempted to be
loaded:
```
internal/bootstrap/loaders.js:189
if (!mod) throw new TypeError(`Missing internal module '${id}'`);
^
TypeError: Missing internal module 'internal/a'
```
PR-URL: https://github.com/nodejs/node/pull/29593
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Since the bootstrap does not actually use ESM at all, there
is no need to create this map so early. This patch moves
the initialization of the map to pre-execution,
so that the only binding loaded in loaders is native_module.
In addition, switch to SafeWeakMap.
PR-URL: https://github.com/nodejs/node/pull/27323
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
This changes the primordials to expose built-in prototypes with their
methods already uncurried.
The uncurryThis function is therefore moved to the primordials.
All uses of uncurryThis on built-ins are changed to import the relevant
prototypes from primordials.
All uses of Function.call.bind are also changed to use primordials.
PR-URL: https://github.com/nodejs/node/pull/27096
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
This patch removes the redundant `require-globals` custom
eslint rule by removing `env: node` in the eslint config
and whitelist the globals that can be accessed in native
modules instead of black listing them. This makes sense
for our `lib/` files because here we are creating the
Node.js environment instead of running in a normal user
land Node.js environment.
PR-URL: https://github.com/nodejs/node/pull/27082
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Instead of relying on the value of the CLI option when
executing bootstrap/loaders.js.
PR-URL: https://github.com/nodejs/node/pull/26759
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
This patch removes `NativeModule.require` and
`NativeModule.requireWithFallbackInDeps`. The public loaders now
have to use a special method
`NativeModule.prototype.compileForPublicLoader()` to compile native
modules. In addition this patch moves the decisions of proxifying
exports and throwing unknown builtin errors entirely to public
loaders, and skip those during internal use - therefore `loaders.js`,
which is compiled during bootstrap, no longer needs to be aware of
the value of `--experimental-modules`.
PR-URL: https://github.com/nodejs/node/pull/26549
Reviewed-By: James M Snell <jasnell@gmail.com>
This patch moves the dispatch of `Profiler.takePreciseCoverage`
to a point before the bootstrap scripts are run to ensure that
we can collect coverage data for all the scripts run after
the inspector agent is ready.
Before this patch `lib/internal/bootstrap/primordials.js` was not
covered by `make coverage`, after this patch it is.
PR-URL: https://github.com/nodejs/node/pull/26006
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Coe <bencoe@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
There are no non-internal builtin modules left, so this
should be safe to remove to a large degree.
PR-URL: https://github.com/nodejs/node/pull/25829
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This patches changes the `safe_globals` internal module into a
script that gets run during bootstrap and saves JavaScript builtins
(primordials) into an object that is available for all other builtin
modules to access lexically later.
PR-URL: https://github.com/nodejs/node/pull/25816
Refs: https://github.com/nodejs/node/issues/18795
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Gus Caplan <me@gus.host>
This patch splits the execution mode selection from the environment
setup in `lib/internal/bootstrap/node.js`, and split the entry point
of different execution mode into main scripts under
`lib/internal/main`:
- `check_syntax.js`: used when `-c`/`--check` which only checks the
syntax of the input instead of executing it.
- `eval_stdin.js`: used when `-e` is passed without value and stdin
is not a TTY (e.g. something is piped).
- `eval_string`: used when `-e` is passed along with a string argument
- `inspect.js`: for `node inspect`/`node debug`
- `print_bash_completion.js`: for `--completion-bash`
- `print_help.js`: for `--help`
- `prof_process.js`: for `--prof-process`
- `repl.js`: for the REPL
- `run_main_module.js`: used when a main module is passed
- `run_third_party_main.js`: for the legacy `_third_party_main.js`
support
- `worker_thread.js`: for workers
This makes the entry points easier to navigate and paves the way
for customized v8 snapshots (that do not need to deserialize
execution mode setup) and better embedder APIs.
As an example, after this patch, for the most common case where
Node.js executes a user module as an entry point, it essentially
goes through:
- `lib/internal/per_context.js` to setup the v8 Context (which is
also run when setting up contexts for the `vm` module)
- `lib/internal/bootstrap/loaders.js` to set up internal binding
and builtin module loaders (that are separate from the loaders
accessible in the user land).
- `lib/internal/bootstrap/node.js`: to set up the rest of the
environment, including various globals and the process object
- `lib/internal/main/run_main_module.js`: which is selected from
C++ to prepare execution of the user module.
This patch also removes `NativeModuleLoader::CompileAndCall` and
exposes `NativeModuleLoader::LookupAndCompile` directly so that
we can handle syntax errors and runtime errors of bootstrap
scripts differently.
PR-URL: https://github.com/nodejs/node/pull/25667
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Instead of using `internalBinding('config')` which should be used
to carry information about build-time options, directly pass the
run-time cli options into bootstrap/loaders.js lexically via
function arguments.
PR-URL: https://github.com/nodejs/node/pull/25463
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Refael Ackermann <refack@gmail.com>
- Renamed `internal/process/write-coverage.js` to
`internal/coverage-gen/with_instrumentation.js`,
`internal/process/coverage.js` to
`internal/coverage-gen/with_profiler.js` to distinguish
the two better and added comments.
- Separate the coverage directory setup and the connection
setup, moves the directory setup into `node.js` and
closer to the exit hooks because that's where it's used.
- Moves the `process.reallyExit` overwrite and
`process.on('exit')` hooks setup into bootstrap/node.js
for clarity, and move them to a later stage of
bootstrap since they do not have to happen that early.
PR-URL: https://github.com/nodejs/node/pull/25398
Reviewed-By: Ben Coe <bencoe@gmail.com>
- Remove `NativeModule._source` - the compilation is now entirely
done in C++ and `process.binding('natives')` is implemented
directly in the binding loader so there is no need to store
additional source code strings.
- Instead of using an object as `NativeModule._cached` and insert
into it after compilation of each native module, simply prebuild
a JS map filled with all the native modules and infer the
state of compilation through `mod.loading`/`mod.loaded`.
- Rename `NativeModule.nonInternalExists` to
`NativeModule.canBeRequiredByUsers` and precompute that
property for all the native modules during bootstrap instead
of branching in every require call during runtime. This also fixes
the bug where `worker_threads` can be made available with
`--expose-internals`.
- Rename `NativeModule.requireForDeps` to
`NativeModule.requireWithFallbackInDeps`.
- Add a test to make sure we do not accidentally leak any module
to the global namespace.
PR-URL: https://github.com/nodejs/node/pull/25352
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Having an experimental feature behind a flag makes change
if we are expecting significant breaking changes to its API.
Since the Worker API has been essentially stable since
its initial introduction, and no noticeable doubt about
possibly not keeping the feature around has been voiced,
removing the flag and thereby reducing the barrier to experimentation,
and consequently receiving feedback on the implementation,
seems like a good idea.
PR-URL: https://github.com/nodejs/node/pull/25361
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Yuta Hiroto <hello@hiroppy.me>
Reviewed-By: Shingo Inoue <leko.noor@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Masashi Hirano <shisama07@gmail.com>
Reviewed-By: Weijia Wang <starkwang@126.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
This brings DEP0084 to End-of-Life. It is unlikely that this
has received much public usage in the first place, so removing
should be okay.
PR-URL: https://github.com/nodejs/node/pull/25138
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
