Commit Graph

79 Commits

Author SHA1 Message Date
Moshe Atlow
ca033c16fe
test_runner: align behavior of it and test
PR-URL: https://github.com/nodejs/node/pull/46889
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-03-05 09:42:29 +02:00
Moshe Atlow
a37b72da87
test_runner: avoid running twice tests in describe
PR-URL: https://github.com/nodejs/node/pull/46888
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-03-03 07:27:16 +02:00
Moshe Atlow
c7e6a0c1b1
test_runner: fix reconstruction of errors extracted from YAML
PR-URL: https://github.com/nodejs/node/pull/46872
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Nitzan Uziely <linkgoron@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-03-01 22:16:39 +00:00
Moshe Atlow
9c61d414c4
test_runner: reset count on watch mode
PR-URL: https://github.com/nodejs/node/pull/46577
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-26 16:28:30 +02:00
Colin Ihrig
0c90be9072
test_runner: better handle async bootstrap errors
The test runner is bootstrapped synchronously, with the exception
of importing custom reporters. To better handle asynchronously
throw errors, this commit introduces an internal error type that
can be checked for from the test runner's uncaughtException
handler.

After https://github.com/nodejs/node/pull/46707 and this change
land, the other throw statement in the uncaughtException handler
can be removed. This will allow the test runner to handle errors
thrown from outside of tests (which currently prevents the test
runner from reporting results).

PR-URL: https://github.com/nodejs/node/pull/46720
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-02-24 14:13:24 +00:00
Richie McColl
0093fd3ca8
test_runner: add describe.only and it.only shorthands
PR-URL: https://github.com/nodejs/node/pull/46604
Fixes: https://github.com/nodejs/node/issues/46562
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-02-21 22:28:03 +00:00
Moshe Atlow
ce49f79ed5
test_runner: bootstrap reporters before running tests
PR-URL: https://github.com/nodejs/node/pull/46737
Fixes: https://github.com/nodejs/node/issues/46747
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-02-21 23:06:13 +02:00
Richie McColl
c90ea9301b
test_runner: emit test-only diagnostic warning
PR-URL: https://github.com/nodejs/node/pull/46540
Fixes: https://github.com/nodejs/node/issues/46448
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-21 08:39:17 +00:00
Colin Ihrig
5e954c3ce3
test_runner: centralize CLI option handling
The test runner relies on a few CLI options. That code was spread
across a few locations. This commit centralizes that logic.

PR-URL: https://github.com/nodejs/node/pull/46707
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-20 01:26:04 +00:00
Richie McColl
261bf9652b
test_runner: display skipped tests in spec reporter output
PR-URL: https://github.com/nodejs/node/pull/46651
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-02-18 22:36:08 +00:00
Mert Can Altın
3c6547fcc8
test_runner: parse non-ascii character correctly
PR-URL: https://github.com/nodejs/node/pull/45736
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-02-18 18:10:16 +00:00
Moshe Atlow
3354f89dcf
test_runner: flatten TAP output when running using --test
PR-URL: https://github.com/nodejs/node/pull/46440
Fixes: https://github.com/nodejs/node/issues/45833
Refs: https://github.com/nodejs/node/issues/45833
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-02-18 16:19:24 +00:00
Moshe Atlow
2787e2dfc2
test_runner: allow nesting test within describe
PR-URL: https://github.com/nodejs/node/pull/46544
Fixes: https://github.com/nodejs/node/issues/46478
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-17 13:42:01 +00:00
Antoine du Hamel
cbcefdf253
lib: enforce use of trailing commas in more files
Some part of the codebase already use trailing commas, this commit is
adding a lint rule to ensure it stays this way.
This commit also adds the rule for a few files that were missing only
one or two trailing commas.

PR-URL: https://github.com/nodejs/node/pull/46655
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-02-16 18:47:24 +01:00
Antoine du Hamel
fe514bf960
lib: enforce use of trailing commas for functions
PR-URL: https://github.com/nodejs/node/pull/46629
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
2023-02-14 18:45:16 +01:00
Pulkit Gupta
4c08c20e57
test: fix tap parser fails if a test logs a number
PR-URL: https://github.com/nodejs/node/pull/46056
Fixes: https://github.com/nodejs/node/issues/46048
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-02-06 16:58:51 +00:00
Moshe Atlow
1118db718c
test_runner: fix missing test diagnostics
PR-URL: https://github.com/nodejs/node/pull/46450
Fixes: https://github.com/nodejs/node/issues/45911
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-02-02 20:00:06 +00:00
Pulkit Gupta
61c65b066b
test_runner: top-level diagnostics not ommited when running with --test
PR-URL: https://github.com/nodejs/node/pull/46441
Fixes: https://github.com/nodejs/node/issues/45910
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-02-02 10:42:43 +00:00
Pulkit Gupta
2f38c74e26
test: fix tap escaping with and without --test
PR-URL: https://github.com/nodejs/node/pull/46311
Fixes: https://github.com/nodejs/node/issues/45836
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-01-28 21:07:47 +00:00
Antoine du Hamel
931f035bac test_runner: update comment to comply with eslint no-fallthrough rule
PR-URL: https://github.com/nodejs/node/pull/46258
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2023-01-19 16:47:39 +00:00
Jordan Harband
757c104147
tools: add prefer-proto rule
fixup: add support for `Object.create(null)`

fixup: extend to any 1-argument Object.create call

fixup: add tests
PR-URL: https://github.com/nodejs/node/pull/46083
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-01-10 05:38:36 +00:00
cjihrig
1ecc03eccd
doc,lib,src,test: rename --test-coverage
Add experimental to the name as requested during review.

PR-URL: https://github.com/nodejs/node/pull/46017
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
2023-01-08 16:10:00 -05:00
cjihrig
8c95cb012b
test_runner: add initial code coverage support
This commit adds code coverage functionality to the node:test
module. When node:test is used in conjunction with the new
--test-coverage CLI flag, a coverage report is created when
the test runner finishes. The coverage summary is forwarded to
any test runner reporters so that the display can be customized
as desired. This new functionality is compatible with the
existing NODE_V8_COVERAGE environment variable as well.

There are still several limitations, which will be addressed in
subsequent pull requests:

- Coverage is only reported for a single process. It is possible
  to merge coverage reports together. Once this is done, the
  --test flag will be supported as well.
- Source maps are not currently supported.
- Excluding specific files or directories from the coverage
  report is not currently supported. Node core modules and
  node_modules/ are excluded though.

PR-URL: https://github.com/nodejs/node/pull/46017
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
2023-01-08 16:09:41 -05:00
Colin Ihrig
4788c0d2a0
test_runner: make built in reporters internal
This commit updates the test runner to make the built in test
reporters internal modules.

PR-URL: https://github.com/nodejs/node/pull/46092
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-01-06 19:34:12 +00:00
Moshe Atlow
9eb363a3e0
test_runner: report file in test runner events
PR-URL: https://github.com/nodejs/node/pull/46030
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-01-02 21:22:54 +00:00
Colin Ihrig
fe5710effc
test_runner: use os.availableParallelism()
This commit addresses an existing TODO in the code by moving
to the new os.availableParallelism() instead of os.cpus().length.

PR-URL: https://github.com/nodejs/node/pull/45969
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-12-27 03:30:19 +00:00
Geoffrey Booth
12c0571c8f
module: move test reporter loading
Move the logic for handling --test-reporter out of the
general module loader and into the test_runner subsystem.

PR-URL: https://github.com/nodejs/node/pull/45923
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-12-22 21:40:32 +00:00
Moshe Atlow
a1b27b25bb
test_runner: add reporters
PR-URL: https://github.com/nodejs/node/pull/45712
Fixes: https://github.com/nodejs/node/issues/45648
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2022-12-19 17:35:57 +00:00
Erick Wendel
929aada39d
test: fix mock.method to support class instances
It fixes a problem when trying to spy a method
from a class instance or static functions
on a class instance

PR-URL: https://github.com/nodejs/node/pull/45608
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-12-17 14:35:09 +00:00
Colin Ihrig
385d595a4f
test_runner: run t.after() if test body throws
This commit fixes a bug where t.after() was not called if the
test body threw an exception.

PR-URL: https://github.com/nodejs/node/pull/45870
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-12-17 06:36:15 +00:00
Moshe Atlow
232efb06fe
test_runner: parse yaml
PR-URL: https://github.com/nodejs/node/pull/45815
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-12-13 14:51:35 +00:00
Antoine du Hamel
2483da743c
test_runner: refactor tap_lexer to use more primordials
PR-URL: https://github.com/nodejs/node/pull/45744
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2022-12-12 07:30:47 +01:00
Antoine du Hamel
7a42a206ac
test_runner: refactor tap_parser to use more primordials
PR-URL: https://github.com/nodejs/node/pull/45745
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2022-12-11 22:34:31 +00:00
Pulkit Gupta
22dc987fde
test: fix invalid output TAP if there newline in test name
PR-URL: https://github.com/nodejs/node/pull/45742
Fixes: https://github.com/nodejs/node/issues/45396
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2022-12-11 20:02:37 +00:00
cjihrig
215c5317d4 test_runner: add t.after() hook
This commit adds an after() hook to the TestContext class. This
hook can be used to clean up after a test finishes.

PR-URL: https://github.com/nodejs/node/pull/45792
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-12-11 19:19:07 +00:00
cjihrig
8302b0add0 test_runner: don't use a symbol for runHook()
This is not exposed to userland, so there is no need to put it
behind a symbol.

PR-URL: https://github.com/nodejs/node/pull/45792
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-12-11 19:19:06 +00:00
MURAKAMI Masahiko
01b7ac6051
test_runner: add resetCalls to MockFunctionContext
This commit allows tests in test runner to reset
the calls of mock function

Refs: https://github.com/nodejs/node/pull/45326#discussion_r1014728750
PR-URL: https://github.com/nodejs/node/pull/45710
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2022-12-08 20:28:38 +00:00
Debadree Chatterjee
8541b3a4c0
lib: added SuiteContext class
added SuiteContext class to replace object literal

Fixes: https://github.com/nodejs/node/issues/45641
Refs: https://github.com/nodejs/node/issues/45641#issuecomment-1329130581
PR-URL: https://github.com/nodejs/node/pull/45687
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2022-12-02 06:29:48 +00:00
Colin Ihrig
cc2732d764
test_runner: don't parse TAP from stderr
This commit stops the test runner CLI from parsing child
process stderr as TAP. Per the TAP spec, TAP can only come from
stdout. To avoid losing stderr data, those logs are injected
into the parser as unknown tokens so that they are output as
comments.

PR-URL: https://github.com/nodejs/node/pull/45618
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2022-12-02 03:12:07 +00:00
MURAKAMI Masahiko
afed1afa55 test_runner: add getter and setter to MockTracker
This commit allows tests in test runner to use the
`getter` and `setter` methods as "syntax sugar" for
`MockTracker.method` with the `options.getter` or
`options.setter` set to true in the options.

Refs: https://github.com/nodejs/node/pull/45326#discussion_r1014727289
PR-URL: https://github.com/nodejs/node/pull/45506
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-11-27 23:12:13 +00:00
Colin Ihrig
fec0fbc333
test_runner: remove stdout and stderr from error
The CLI test runner parses the TAP output from child processes
now and displays it. This commit removes a previous workaround
for displaying child process stdout and stderr when tests
failures occurred.

PR-URL: https://github.com/nodejs/node/pull/45592
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2022-11-25 02:48:21 +00:00
Wassim Chegham
f8ce9117b1
test_runner: add initial TAP parser
Work in progress

PR-URL: https://github.com/nodejs/node/pull/43525
Refs: https://github.com/nodejs/node/issues/43344
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2022-11-21 22:50:21 +00:00
Moshe Atlow
97355c74e8
test_runner: support watch mode
PR-URL: https://github.com/nodejs/node/pull/45214
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-11-13 21:40:26 +00:00
cjihrig
7c6682957b
test_runner: support function mocking
This commit allows tests in the test runner to mock functions
and methods.

PR-URL: https://github.com/nodejs/node/pull/45326
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-11-07 18:25:54 -05:00
MURAKAMI Masahiko
06603c44a5
test_runner: avoid swallowing of asynchronously thrown errors
Fixes: https://github.com/nodejs/node/issues/44612
PR-URL: https://github.com/nodejs/node/pull/45264
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2022-11-07 12:02:56 +00:00
Jithil P Ponnan
3759935ee2
test_runner: fix afterEach not running on test failures
test_runner: fix afterEach not running on test failures

PR-URL: https://github.com/nodejs/node/pull/45204
Fixes: https://github.com/nodejs/node/issues/45192
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2022-11-07 05:58:39 +00:00
Moshe Atlow
3e57891ee2
test_runner: report tap subtest in order
PR-URL: https://github.com/nodejs/node/pull/45220
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2022-10-30 15:36:35 +02:00
Colin Ihrig
a69a30016c
test_runner: call {before,after}Each() on suites
Prior to this commit, beforeEach() and afterEach() hooks were
not called on test suites (describe()). This commit addresses that.

Fixes: https://github.com/nodejs/node/issues/45028
PR-URL: https://github.com/nodejs/node/pull/45161
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2022-10-27 08:27:35 +00:00
Bryan English
e260b373f1
test_runner: add extra fields in AssertionError YAML
Many TAP reporters offer special-case handling of YAML objects
containing `expected`, `actual`, and `operator` fields, as produced by
`AssertionError` and similar userland Error classes.

PR-URL: https://github.com/nodejs/node/pull/44952
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: James M Snell <jasnell@gmail.com>
2022-10-26 04:00:13 +00:00
Joyee Cheung
be525d7d04
src: consolidate exit codes in the code base
Add an ExitCode enum class and use it throughout the code base
instead of hard-coding the exit codes everywhere. At the moment,
the exit codes used in many places do not actually conform to
what the documentation describes. With the new enums (which
are also available to the JS land as constants in an internal
binding) we could migrate to a more consistent usage of the
codes, and eventually expose the constants to the user land
when they are stable enough.

PR-URL: https://github.com/nodejs/node/pull/44746
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
2022-10-06 12:41:16 +00:00