This fixes the necessary plumbing to make the --shell argument
have its intended effect.
PR-URL: https://github.com/nodejs/node/pull/26449
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
This way we can specify a custom path for the test folder, e.g.
when building addons separately from the source tree.
PR-URL: https://github.com/nodejs/node/pull/26093
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Instead of using process.config.variables.v8_enable_inspector
to detect whether inspector is enabled in the build.
PR-URL: https://github.com/nodejs/node/pull/25819
Refs: https://github.com/nodejs/node/issues/25343
Reviewed-By: Eugene Ostroukhov <eostroukhov@google.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Enable marking of coverage tests so that we can
allow some tests to fail without blocking the generation
of coverage data. This will later allow us to
fail the coverage job if other kinds of errors occur and
to capture which tests we believe are not running properly
with coverage enabled.
PR-URL: https://github.com/nodejs/node/pull/25671
Reviewed-By: Ben Coe <bencoe@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Having an experimental feature behind a flag makes change
if we are expecting significant breaking changes to its API.
Since the Worker API has been essentially stable since
its initial introduction, and no noticeable doubt about
possibly not keeping the feature around has been voiced,
removing the flag and thereby reducing the barrier to experimentation,
and consequently receiving feedback on the implementation,
seems like a good idea.
PR-URL: https://github.com/nodejs/node/pull/25361
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Yuta Hiroto <hello@hiroppy.me>
Reviewed-By: Shingo Inoue <leko.noor@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Masashi Hirano <shisama07@gmail.com>
Reviewed-By: Weijia Wang <starkwang@126.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Some tests which create files and check file permissions assume the
umask is compatible with 022, and break when set to something like 007.
Explicitly set umask to 022
PR-URL: https://github.com/nodejs/node/pull/25213
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Partition test/addons-napi into test/js-native-api and test/node-api to
isolate the Node.js-agnostic portion of the N-API tests from the
Node.js-specific portion.
PR-URL: https://github.com/nodejs/node/pull/24557
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
While running the test suite the progress bar shows former line
endings if the new line is shorter than the former line. The length
was calculated without the line ending. It is now an empty string
to prevent the off by one error instead of using extra whitespace.
PR-URL: https://github.com/nodejs/node/pull/24748
Refs: https://github.com/nodejs/node/pull/24486
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Currently, configuring --without-ssl will cause the following test to
fail:
=== release test-https-agent-additional-options ===
Path: parallel/test-https-agent-additional-options
out/Release/node: bad option: --tls-v1.1
Command: out/Release/node --tls-v1.1
/node/test/parallel/test-https-agent-additional-options.js
=== release test-https-agent-session-eviction ===
Path: parallel/test-https-agent-session-eviction
out/Release/node: bad option: --tls-v1.0
Command: out/Release/node --tls-v1.0
/node/test/parallel/test-https-agent-session-eviction.js
This commit adds a check for the --tls-v.x flags and skips them if node
was built without crypto support.
PR-URL: https://github.com/nodejs/node/pull/24376
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Move benchmark tests (which are slow) out of the main test suite. We can
hopefully add them to node-daily-master so that they are still run daily
on CI.
PR-URL: https://github.com/nodejs/node/pull/24265
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
- Add the code cache tests to the default test suite, and test
the bookkeeping when the binary is not built with the code cache.
- Test the code cache generator to make sure we do not accidentally
break it - until we enable code cache in the CI.
Refs: https://github.com/nodejs/node/issues/21563
PR-URL: https://github.com/nodejs/node/pull/23855
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Ref: https://github.com/nodejs/node/pull/23585#issuecomment-430585490
Python's `list.remove` will throw if the element is not found and also
it removes only the first occurrence.
This patch replaces the use of `list.remove` with a `filter` which
solves both of the above mentioned problems.
PR-URL: https://github.com/nodejs/node/pull/23727
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Matheus Marchini <mat@mmarchini.me>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
* Hoist common code to base class
(`GetTestStatus`, and the `section` property to `TestConfiguration`)
* Replace ListSet with the built in set
* Remove ClassifiedTest
* Inline PrintReport
* How cases_to_run are filtered
PR-URL: https://github.com/nodejs/node/pull/23251
Reviewed-By: Rich Trott <rtrott@gmail.com>
Since faking TTY input is not otherwise fake-able, we need
support in the test runner for it.
PR-URL: https://github.com/nodejs/node/pull/23053
Reviewed-By: James M Snell <jasnell@gmail.com>
There’s no reason to have a separate addon just for
testing GC anymore.
PR-URL: https://github.com/nodejs/node/pull/22001
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jon Moss <me@jonathanmoss.me>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Removes a couple of unused/empty functions inside of `tools/test.py`
PR-URL: https://github.com/nodejs/node/pull/22010
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
- Updates `test/README.md` with new suites
- Fixes some outdated `IGNORED_SUITES` listings
- Allows for `test/known_issues` suite to be run by default
PR-URL: https://github.com/nodejs/node/pull/21910
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Lance Ball <lball@redhat.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
This patch speeds up the startup time and reduce the startup memory
footprint by using V8 code cache when comiling builtin modules.
The current approach is demonstrated in the `with-code-cache`
Makefile target (no corresponding Windows target at the moment).
1. Build the binary normally (`src/node_code_cache_stub.cc` is used),
by now `internalBinding('code_cache')` is an empty object
2. Run `tools/generate_code_cache.js` with the binary, which generates
the code caches by reading source code of builtin modules off source
code exposed by `require('internal/bootstrap/cache').builtinSource`
and then generate a C++ file containing static char arrays of the
code cache, using a format similar to `node_javascript.cc`
3. Run `configure` with the `--code-cache-path` option so that
the newly generated C++ file will be used when compiling the
new binary. The generated C++ file will put the cache into
the `internalBinding('code_cache')` object with the module
ids as keys
4. The new binary tries to read the code cache from
`internalBinding('code_cache')` and use it to compile
builtin modules. If the cache is used, it will put the id
into `require('internal/bootstrap/cache').compiledWithCache`
for bookkeeping, otherwise the id will be pushed into
`require('internal/bootstrap/cache').compiledWithoutCache`
This patch also added tests that verify the code cache is
generated and used when compiling builtin modules.
The binary with code cache:
- Is ~1MB bigger than the binary without code cahe
- Consumes ~1MB less memory during start up
- Starts up about 60% faster
PR-URL: https://github.com/nodejs/node/pull/21405
Reviewed-By: John-David Dalton <john.david.dalton@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Gus Caplan <me@gus.host>
`v8-updates` holds all tests related to V8 updates, for example, testing
for postmortem metadata chages.
PR-URL: https://github.com/nodejs/node/pull/20783
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Enable running tests inside workers by passing `--worker`
to `tools/test.py`. A number of tests are marked as skipped,
or have been slightly altered to fit the different environment.
PR-URL: https://github.com/nodejs/node/pull/20876
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Shingo Inoue <leko.noor@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: John-David Dalton <john.david.dalton@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Make test runner capable of skipping tests, which makes it possible
to skip the failing test/message/core_line_numbers.js test.
Make nyc no longer generate compact instrumentation (this causes
significantly different code output, which leads to failing test
assertions).
PR-URL: https://github.com/nodejs/node/pull/20794
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
When a test times out, the contents of stdout and stderr can often be
highly valuable in debugging. Provide that information.
Refs: https://github.com/nodejs/node/issues/19906#issuecomment-382073888
PR-URL: https://github.com/nodejs/node/pull/20260
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Include the exit code in test failures. This will give us more
information during the currently-puzzling failures that provide no
information in CI such as:
```
03:10:10 not ok 563 parallel/test-fs-truncate
03:10:10 ---
03:10:10 duration_ms: 1.119
03:10:10 severity: fail
03:10:10 stack: |-
03:10:10 ...
```
PR-URL: https://github.com/nodejs/node/pull/19855
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
common.PORT should not be used in parallelized tests. (There can be a
port collision if another tests requests an arbitrary open port from the
operating system and ends up getting common.PORT before a test that uses
common.PORT uses the port.) In such a situation, throw an error.
PR-URL: https://github.com/nodejs/node/pull/17559
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Adds a way to mark a specified test as 'flaky' on fips compliant
systems.
Earlier, the ``tools/test.py`` script supported only 'mode',
'system' and 'arch' for test environment specification. This limits the
ability to specify the behavior of tests and setting pre-determined
behavior of the same on other types of systems. As an example, the
feature request below indicates the need to specify certain tests as
'flaky' on fips compliant systems. It hints at future possibility of a
shared library, which in turn may need a specifier for running tests.
This commit introduces a new item in the ``env`` dict, called ``type``
which defaults to ``simple`` type. It also adds an optional command
line argument ``--type``, which inputs strings. Current functionality
extends to setting ``simple`` or ``fips`` for this ``type`` variable.
However, extending it to further uses is rather simple by adding "if"
conditions at appropriate places in the ``tools/test.py`` script.
PR-URL: https://github.com/nodejs/node/pull/16329
Fixes: https://github.com/nodejs/node/issues/14746
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
`test-make-doc` fails in CI on Raspberry Pi devices where `test-ci-js`
runs the test but does not build the docs. We do not want to build the
docs in these cases. Move the test to the `doctool` suite and add that
suite to `IGNORED_SUITES` in `test.py`. Specify `doctool` as a test
suite to run with `make test` or `make test-ci` but not with the
`make test-ci-js` job run on cross-compiled fanned CI tests like we do
with Raspberry Pi devices in CI.
PR-URL: https://github.com/nodejs/node/pull/16301
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
The 'Main' function in tools/test.py file was using a variable named
``all_outcomes`` to store a value not being used. It is a best practice
to name unused variables, often return values of functions/methods (as
in this case) as ``_`` [1]. This just helps keep the code a bit cleaner
and avoid any silly mistakes.
PR-URL: https://github.com/nodejs/node/pull/16171
Refs: [1] https://stackoverflow.com/a/5477153
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Claudio Rodriguez <cjrodr@yahoo.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
You can now run suites for subsystem using shorthand, e.g., http.
Switch to black-list of default test folders from white-list.
Tests run by 'make test', 'make coverage', etc., now configurable.
Stop running known_issues suite when collecting test coverage.
PR-URL: https://github.com/nodejs/node/pull/15450
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
When ESM support was added it created a regression in the test runner
that broke the ability to run individual tests. This commit
re-introduces the use of `NormalizePath` which fixes the regression
in the test runner
Refs: https://github.com/nodejs/node/pull/15300
PR-URL: https://github.com/nodejs/node/pull/15329
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This follows the EPS an allows the node CLI to have ESM as an entry point.
`node ./example.mjs`. A newer V8 is needed for `import()` so that is not
included. `import.meta` is still in specification stage so that also is not
included.
PR-URL: https://github.com/nodejs/node/pull/14369
Author: Bradley Farias <bradley.meck@gmail.com>
Author: Guy Bedford <guybedford@gmail.com>
Author: Jan Krems <jan.krems@groupon.com>
Author: Timothy Gu <timothygu99@gmail.com>
Author: Michaël Zasso <targos@protonmail.com>
Author: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
