Commit Graph

58 Commits

Author SHA1 Message Date
Colin Ihrig
034a2409bf
test_runner: remove plan option from run()
This commit removes the plan option to run(). I believe it was
added by mistake. It is not documented, untested, and a test
plan does not make sense in the context of run().

This commit also fixes a minor formatting issue in a related
fixture.

Refs: https://github.com/nodejs/node/pull/52860
PR-URL: https://github.com/nodejs/node/pull/53834
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2024-07-15 17:10:01 +00:00
jakecastelli
7125931f4c
test_runner: handle file rename and deletion under watch mode
Fixes: https://github.com/nodejs/node/issues/53113
PR-URL: https://github.com/nodejs/node/pull/53114
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
2024-06-02 08:11:30 +00:00
Deokjin Kim
88d39524b1
test_runner: refactor to use min/max of validateInteger
Instead of additional `if` statement, use min/max of
`validateInteger` for `shard.index`.

PR-URL: https://github.com/nodejs/node/pull/53148
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2024-06-01 22:21:25 +00:00
Moshe Atlow
c1657510b5
test_runner: fix watch mode race condition
PR-URL: https://github.com/nodejs/node/pull/52954
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-05-14 08:31:02 +00:00
Colin Ihrig
0c83f80ff3 test_runner: support test plans
Co-Authored-By: Marco Ippolito <marcoippolito54@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/52860
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
2024-05-09 12:09:09 +02:00
Antoine du Hamel
231548b5cf
lib: enforce ASCII order in error code imports
PR-URL: https://github.com/nodejs/node/pull/52625
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-04-23 17:05:38 +00:00
Antoine du Hamel
a596af0819
tools: add lint rule to keep primordials in ASCII order
PR-URL: https://github.com/nodejs/node/pull/52592
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2024-04-21 16:53:08 +00:00
Aviv Keller
e9c233cd6b
test_runner: add --test-skip-pattern cli option
PR-URL: https://github.com/nodejs/node/pull/52529
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-04-18 20:59:50 +00:00
Moshe Atlow
433bd1b04d
test_runner: fix recursive run
PR-URL: https://github.com/nodejs/node/pull/52322
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2024-04-04 13:10:52 +00:00
Moshe Atlow
f9755f6f79
test_runner: emit diagnostics when watch mode drains
PR-URL: https://github.com/nodejs/node/pull/52130
Fixes: https://github.com/nodejs/node/issues/51253
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2024-03-19 21:55:25 +00:00
Colin Ihrig
05db979c01
test_runner: run top level tests in a microtask
This commit updates the test harness to prevent top level
tests from executing immediately. This allows certain config
data, such as filtering options, to be discovered before running
the tests.

PR-URL: https://github.com/nodejs/node/pull/52092
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-03-17 04:17:36 +00:00
Colin Ihrig
84de97a61e
test_runner: support forced exit
This commit updates the test runner to allow a forced exit once
all known tests have finished running.

Fixes: https://github.com/nodejs/node/issues/49925
PR-URL: https://github.com/nodejs/node/pull/52038
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2024-03-13 10:49:15 +00:00
Pulkit Gupta
dad666ad06
test_runner: fixed test object is incorrectly passed to setup()
PR-URL: https://github.com/nodejs/node/pull/50982
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-12-24 08:23:54 +00:00
Pulkit Gupta
f801b58e77
test_runner: fix infinite loop when files are undefined in test runner
PR-URL: https://github.com/nodejs/node/pull/51047
Fixes: https://github.com/nodejs/node/issues/48823
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-12-10 09:35:22 +00:00
Moshe Atlow
3cce03a03f
test_runner: pass abortSignal to test files
PR-URL: https://github.com/nodejs/node/pull/50630
Fixes: https://github.com/nodejs/node/issues/50583
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-11-10 22:09:15 +00:00
Raz Luvaton
527589b755
test_runner: disallow array in run options
PR-URL: https://github.com/nodejs/node/pull/49935
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-10-06 20:21:15 +00:00
Colin Ihrig
4b35a9cfda
test_runner: add test location for FileTests
This commit adds the previously missing test location for
FileTest tests.

Fixes: https://github.com/nodejs/node/issues/49926
Fixes: https://github.com/nodejs/node/issues/49927
PR-URL: https://github.com/nodejs/node/pull/49999
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2023-10-03 18:19:51 +00:00
Moshe Atlow
c9e72e34ab
test_runner: accept testOnly in run
PR-URL: https://github.com/nodejs/node/pull/49753
Fixes: https://github.com/nodejs/node/issues/49733
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
2023-09-24 10:46:13 +00:00
Raz Luvaton
ee391f3781
test_runner: add __proto__ null
PR-URL: https://github.com/nodejs/node/pull/48663
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-07-29 13:22:16 +00:00
Moshe Atlow
48345d0f62 watch: decrease debounce rate
PR-URL: https://github.com/nodejs/node/pull/48926
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-07-27 20:39:14 +00:00
Moshe Atlow
b814de6d77 watch: use debounce instead of throttle
PR-URL: https://github.com/nodejs/node/pull/48926
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-07-27 20:39:13 +00:00
Raz Luvaton
99881304c4
test_runner: add shards support
PR-URL: https://github.com/nodejs/node/pull/48639
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-07-05 03:57:28 +00:00
Moshe Atlow
7cd4e70948 test_runner: support passing globs
PR-URL: https://github.com/nodejs/node/pull/47653
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-06-25 14:40:34 +00:00
Chemi Atlow
78f6952751
fs: use kResistStopPropagation
PR-URL: https://github.com/nodejs/node/pull/48521
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-06-24 15:52:38 +00:00
Moshe Atlow
5d685e4198 test_runner: emit testdrained event
PR-URL: https://github.com/nodejs/node/pull/48259
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-06-02 00:28:09 +00:00
Moshe Atlow
ca71ae6549 test_runner: stop watch mode when abortSignal aborted
PR-URL: https://github.com/nodejs/node/pull/48259
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-06-02 00:28:08 +00:00
Moshe Atlow
e74465d678 test_runner: pass FORCE_COLOR to child process
PR-URL: https://github.com/nodejs/node/pull/48057
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-05-25 10:49:39 +00:00
Moshe Atlow
40a7a055c6 test_runner: dont split lines on test:stdout
PR-URL: https://github.com/nodejs/node/pull/48057
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-05-25 10:49:38 +00:00
Moshe Atlow
f0e70eede2
test_runner: fix test deserialize edge cases
PR-URL: https://github.com/nodejs/node/pull/48106
Fixes: https://github.com/nodejs/node/issues/48103
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-05-23 21:03:19 +00:00
Shiba
1229a22e51
test_runner: delegate stderr and stdout formatting to reporter
Introduce new `TestsStream` events `test:stderr` and `test:stdout`
to delegate `stderr` and `stdout` (e.g. `console.log()`) formatting
to the reporter. And patch existing reporters to:
- Spec: output the message as it is
- TAP: stay the same with existing `test:diagnostic`

PR-URL: https://github.com/nodejs/node/pull/48045
Fixes: https://github.com/nodejs/node/issues/48011
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-05-23 04:44:48 +00:00
Moshe Atlow
7116bc08d7
test_runner: use v8.serialize instead of TAP
PR-URL: https://github.com/nodejs/node/pull/47867
Fixes: https://github.com/nodejs/node/issues/44656
Fixes: https://github.com/nodejs/node/issues/47955
Fixes: https://github.com/nodejs/node/issues/47481
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-05-15 14:27:45 +00:00
cjihrig
ab3a54f2ee
test_runner: remove no-op validation
The code already checks if testNamePatterns is an
array, and converts it to an array if it is not.

PR-URL: https://github.com/nodejs/node/pull/47687
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Deokjin Kim <deokjin81.kim@gmail.com>
2023-04-25 16:03:51 +00:00
Moshe Atlow
dccd25e5bb test_runner: fix test runner concurrency
PR-URL: https://github.com/nodejs/node/pull/47675
Fixes: https://github.com/nodejs/node/issues/47365
Fixes: https://github.com/nodejs/node/issues/47696
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-04-24 22:32:01 +00:00
Moshe Atlow
dac02632a6 test_runner: fix test counting
PR-URL: https://github.com/nodejs/node/pull/47675
Fixes: https://github.com/nodejs/node/issues/47365
Fixes: https://github.com/nodejs/node/issues/47696
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-04-24 22:32:00 +00:00
atlowChemi
c6279690be
test_runner: add testNamePatterns to run api
Accept a `testNamePatterns` value in the `run` fn, and drill those
patterns to the spawned processes.

PR-URL: https://github.com/nodejs/node/pull/47648
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-04-23 15:30:23 +03:00
Moshe Atlow
46a3cff698
test_runner: fix spec skip detection
PR-URL: https://github.com/nodejs/node/pull/47537
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-04-16 07:26:37 +00:00
Moshe Atlow
d1eaded0d1
test_runner: count nested tests
PR-URL: https://github.com/nodejs/node/pull/47094
Fixes: https://github.com/nodejs/node/issues/46762
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-03-21 08:16:38 +00:00
Steve Herzog
334bb179df
test_runner: support defining test reporter in NODE_OPTIONS
Adds --test-reporter and --test-reporter-destination as
allowable options in NODE_OPTIONS. Also adds the CLI flag
--test-child-process to allow forcing the default
test-reporter for inter-process communication.

Fixes: https://github.com/nodejs/node/issues/46484
PR-URL: https://github.com/nodejs/node/pull/46688
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-03-14 16:59:35 +00:00
cjihrig
4e3bc06e65
test_runner: track bootstrapping process
This commit updates the test harness and root test to track
when bootstrapping has completed.

PR-URL: https://github.com/nodejs/node/pull/46962
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2023-03-07 13:59:37 -05:00
Moshe Atlow
9c61d414c4
test_runner: reset count on watch mode
PR-URL: https://github.com/nodejs/node/pull/46577
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-26 16:28:30 +02:00
Moshe Atlow
ce49f79ed5
test_runner: bootstrap reporters before running tests
PR-URL: https://github.com/nodejs/node/pull/46737
Fixes: https://github.com/nodejs/node/issues/46747
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-02-21 23:06:13 +02:00
Moshe Atlow
3354f89dcf
test_runner: flatten TAP output when running using --test
PR-URL: https://github.com/nodejs/node/pull/46440
Fixes: https://github.com/nodejs/node/issues/45833
Refs: https://github.com/nodejs/node/issues/45833
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-02-18 16:19:24 +00:00
Antoine du Hamel
fe514bf960
lib: enforce use of trailing commas for functions
PR-URL: https://github.com/nodejs/node/pull/46629
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
2023-02-14 18:45:16 +01:00
Moshe Atlow
1118db718c
test_runner: fix missing test diagnostics
PR-URL: https://github.com/nodejs/node/pull/46450
Fixes: https://github.com/nodejs/node/issues/45911
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-02-02 20:00:06 +00:00
Pulkit Gupta
61c65b066b
test_runner: top-level diagnostics not ommited when running with --test
PR-URL: https://github.com/nodejs/node/pull/46441
Fixes: https://github.com/nodejs/node/issues/45910
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-02-02 10:42:43 +00:00
cjihrig
1ecc03eccd
doc,lib,src,test: rename --test-coverage
Add experimental to the name as requested during review.

PR-URL: https://github.com/nodejs/node/pull/46017
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
2023-01-08 16:10:00 -05:00
cjihrig
8c95cb012b
test_runner: add initial code coverage support
This commit adds code coverage functionality to the node:test
module. When node:test is used in conjunction with the new
--test-coverage CLI flag, a coverage report is created when
the test runner finishes. The coverage summary is forwarded to
any test runner reporters so that the display can be customized
as desired. This new functionality is compatible with the
existing NODE_V8_COVERAGE environment variable as well.

There are still several limitations, which will be addressed in
subsequent pull requests:

- Coverage is only reported for a single process. It is possible
  to merge coverage reports together. Once this is done, the
  --test flag will be supported as well.
- Source maps are not currently supported.
- Excluding specific files or directories from the coverage
  report is not currently supported. Node core modules and
  node_modules/ are excluded though.

PR-URL: https://github.com/nodejs/node/pull/46017
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
2023-01-08 16:09:41 -05:00
Moshe Atlow
9eb363a3e0
test_runner: report file in test runner events
PR-URL: https://github.com/nodejs/node/pull/46030
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-01-02 21:22:54 +00:00
Moshe Atlow
a1b27b25bb
test_runner: add reporters
PR-URL: https://github.com/nodejs/node/pull/45712
Fixes: https://github.com/nodejs/node/issues/45648
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2022-12-19 17:35:57 +00:00
Moshe Atlow
232efb06fe
test_runner: parse yaml
PR-URL: https://github.com/nodejs/node/pull/45815
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-12-13 14:51:35 +00:00