Commit Graph

44 Commits

Author SHA1 Message Date
Rich Trott
eed8b0525c test: move tick-processor tests to own directory
The tick-processor tests are inherently non-deterministic. They
therefore have false negatives from time to time. They also
sometimes leave extra processes running.

Move them to their own directory until these issues are sorted. Note
that this means that the tests will not be run in CI. Like the inspector
tests and other tests, they will have to be run manually when they are
wanted.

PR-URL: https://github.com/nodejs/node/pull/9506
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
Reviewed-By: Matthew Loring <mattloring@google.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
2016-11-09 19:34:56 -08:00
Rich Trott
03afecd36d test: fix flaky test-child-process-fork-dgram
`test-child-process-fork-dgram` is unreliable on some platforms,
especially FreeBSD and AIX within the project's continuous integration
testing. It has also been observed to be flaky on macos.

* Confirm child has received the server before sending packets
* Close the server instance on the parent or child after receiving a

Refs: https://github.com/nodejs/node/pull/8697
Fixes: https://github.com/nodejs/node/issues/8949
Fixes: https://github.com/nodejs/node/issues/8271
PR-URL: https://github.com/nodejs/node/pull/9098
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2016-10-17 19:30:01 -07:00
Rich Trott
d097311a1b test: mark test-tick-processor-unknown flaky
Mark test-tick-processor-unknown flaky on SmartOS.

PR-URL: https://github.com/nodejs/node/pull/8900
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
2016-10-04 15:42:00 -07:00
Rich Trott
e65a2d7ddc test: accept expected AIX result test-stdio-closed
AIX handles closed stdio differently (but still compliant with spec as
far as I can tell) than other POSIX variants we test. Test results are
different than Linux and others because AIX takes measures to not re-use
the file descriptors for stdio if one of the stdio streams is closed.

Fixes: https://github.com/nodejs/node/issues/8375
PR-URL: https://github.com/nodejs/node/pull/8755
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com>
2016-09-26 07:55:44 -07:00
Rich Trott
d196c5d2af test: skip cpu-intensive tests on slow hosts
The `test-tick-processor-*` tests are now passing everywhere except for
the single-processor 700MHz Raspberry Pi 1 devices.

The tests are CPU-intensive. Skip the tests if there is only one CPU and
it runs at a speed not more than 700 MHz.

PR-URL: https://github.com/nodejs/node/pull/8652
Reviewed-By: Matthew Loring <mattloring@google.com>
2016-09-24 15:43:56 -07:00
Julien Gilli
7926886bf3 test: fix test-debug-signal-cluster.js flakyness
Do not assume any order and buffering/atomicity of output from child
processes' debugger agents.

Fixes: https://github.com/nodejs/node/issues/3796
PR-URL: https://github.com/nodejs/node/pull/8568
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com>
2016-09-16 15:53:11 -07:00
Fedor Indutny
5a171398f4 test: make test-tick-processor.js non-flaky
Wait for a sought-for symbol to appear instead of just hard-killing
subprocesses at 2s timeout.

Fix: #4427
PR-URL: https://github.com/nodejs/node/pull/8542
Reviewed-By: Rich Trott <rtrott@gmail.com>
2016-09-16 14:17:49 +02:00
Rich Trott
ed640ae4cb crypto: make malloc failure check cross-platform
`malloc(0)` may return NULL on some platforms. Do not report
out-of-memory error unless `malloc` was passed a number greater than
`0`.

PR-URL: https://github.com/nodejs/node/pull/8352
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
2016-09-06 09:39:35 -04:00
Michael Dawson
02ed21aa5c test: mark test-child-process-fork-dgram as flaky
PR-URL: https://github.com/nodejs/node/pull/8274
Reviewed-By: joaocgreis - João Reis <reis@janeasystems.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2016-08-26 08:49:19 -04:00
Michael Dawson
a0b3bc56c6 test: mark test failing on AIX as flaky
We now have adequate AIX hardware to add AIX to
the regular regression runs.

However, there are a couple of failing tests even
though AIX was green at one point.  This PR
marks those tests as flaky so that we can add AIX
so that we can spot any new regressions without making
the builds RED

The tests are being worked under the following PRs

- being worked under https://github.com/nodejs/node/pull/7564
test-async-wrap-post-did-throw
test-async-wrap-throw-from-callback
test-crypto-random

- being worked under https://github.com/nodejs/node/issues/7973
test-stdio-closed

- covered by https://github.com/nodejs/node/issues/3796
test-debug-signal-cluster

PR-URL: https://github.com/nodejs/node/pull/8065
Reviewed-By: joaocgreis - João Reis <reis@janeasystems.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
2016-08-11 14:20:13 -04:00
Rich Trott
814b8c3cf7 test: handle IPv6 localhost issues within tests
The issue of hosts that do not resolve `localhost` to `::1` is now
handled within the tests. Remove flaky status for
test-https-connect-address-family and test-tls-connect-address-family.

PR-URL: https://github.com/nodejs/node/pull/7766
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2016-07-20 10:39:37 -07:00
Ben Noordhuis
cfe76f2d6b src: disable stdio buffering
Disable stdio buffering, it interacts poorly with printf() calls from
elsewhere in the program (e.g., any logging from V8.)  Unbreaks among
other things the `--trace_debug_json` switch.

Undoes commit 0966ab99 ("src: force line buffering for stderr"), which
in retrospect is not a proper fix.  Turning on line buffering fixed a
flaky test on SmartOS but the test wasn't failing on other platforms,
where stderr wasn't line-buffered either.  Mark the test flaky again,
it failed once in a run of 333 tries on the smartos-64 buildbot.

Disabling buffering should be safe even when mixed with non-blocking
stdio I/O because libuv goes to great lengths to reopen the tty file
descriptors and falls back to blocking I/O when that fails.

PR-URL: https://github.com/nodejs/node/pull/7610
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Rich Trott <rtrott@gmail.com>
2016-07-11 23:15:34 +02:00
Anna Henningsen
1d75987ff0
Revert "test: mark test-vm-timeout flaky on windows"
This reverts commit f34caa96d1.

PR-URL: https://github.com/nodejs/node/pull/7373
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2016-06-25 05:50:48 +02:00
Rich Trott
f34caa96d1 test: mark test-vm-timeout flaky on windows
PR-URL: https://github.com/nodejs/node/pull/7359
Refs: https://github.com/nodejs/node/issues/6727
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Joao Reis <reis@janeasystems.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
2016-06-22 05:45:06 -07:00
Ben Noordhuis
fb19b800b8 tls,https: respect address family when connecting
Respect the `{ family: 6 }` address family property when connecting to
a remote peer over TLS.

Fixes: https://github.com/nodejs/node/issues/4139
Fixes: https://github.com/nodejs/node/issues/6440
PR-URL: https://github.com/nodejs/node/pull/6654
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2016-05-28 22:49:36 +02:00
Michael Dawson
b1c05871a3 test: exclude new fs watch test for AIX
As per https://github.com/nodejs/node/issues/5085
exclude new test from AIX until we have fixes for
libuv for fs watching on AIX.  Excluding test
so AIX tests are green and we don't miss
other regressions

PR-URL: https://github.com/nodejs/node/pull/5937
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2016-03-28 19:22:54 -04:00
Trevor Norris
5d7265f601 test: prevent flakey test on pi2
Looping rapidly and making new connections causes problems on pi2.
Instead create a new connection when an old connection has already been
made. Running a stress test of 600 times and they all passed.

Fixes: https://github.com/nodejs/node/issues/5302
PR-URL: https://github.com/nodejs/node/pull/5537
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Alexis Campailla <orangemocha@nodejs.org>
2016-03-03 13:51:46 -07:00
Rich Trott
4d6b4c30dd test: remove flaky designation from fixed tests
`test-tls-ticket-cluster` and `test-vm-syntax-error-stderr` are no
longer flaky.

Refs: https://github.com/nodejs/node/issues/2510
Refs: https://github.com/nodejs/node/issues/2660
PR-URL: https://github.com/nodejs/node/pull/5459
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2016-02-29 10:42:57 -08:00
Rich Trott
bbf4621548 test: mitigate flaky test-http-agent
Reduce number of clients from 100 to 16 as Raspberry Pi in CI starts to
exhibit flakiness around 22 or so clients.

Fixes: https://github.com/nodejs/node/issues/5184
PR-URL: https://github.com/nodejs/node/pull/5346
Reviewed-By: James M Snell <jasnell@gmail.com>
2016-02-22 11:15:31 -08:00
Rich Trott
3144ecd0b2 test: remove flaky mark for test-debug-no-context
test-debug-no-context flakiness was worked around in issue 5269 but the
flaky designation was left for the test. This change removes it.

Refs: https://github.com/nodejs/node/pull/5269
PR-URL: https://github.com/nodejs/node/pull/5317
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2016-02-19 14:55:38 -08:00
Alexis Campailla
c05a330b49 test: test-process-getactivehandles is flaky
Ref: https://github.com/nodejs/node/issues/5302

PR-URL: https://github.com/nodejs/node/pull/5303
Reviewed-By: Rich Trott <rtrott@gmail.com>
2016-02-19 01:46:05 +01:00
Michael Dawson
6a85ad9391 test: disable fs watch tests for AIX
fs watch currently needs special configuration on AIX and we
want to improve under https://github.com/nodejs/node/issues/5085.
Tests are disabled so CI can be green and we can spot other
regressions until this work is complete.

test-async-wrap-check-providers does not aim to test fs watch
but part of the test uses it so that part has been skipped for
AIX

PR-URL: https://github.com/nodejs/node/pull/5187
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2016-02-16 14:55:08 -05:00
Rich Trott
9ab19afe0b test: mark test-http-agent flaky
Ref: https://github.com/nodejs/node/issues/5184
PR-URL: https://github.com/nodejs/node/pull/5209
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Roman Klauke <romaaan.git@gmail.com>
2016-02-15 22:05:39 -08:00
Santiago Gimeno
4e4b260b07 test: fix child-process-fork-regr-gh-2847
The test would sometimes time out on some platforms. Take the initial
version of the test and instead of sending 100 handles, just send 2.
It still fails when run with the code before the change was introduced
and passes afterwards.

Remove test from parallel.status.

PR-URL: https://github.com/nodejs/node/pull/5121
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Fixes: https://github.com/nodejs/node/issues/3635
2016-02-09 16:30:20 -08:00
Rich Trott
6e3bccbc03 test: mark flaky tests on Raspberry Pi
A few tests have started failing on Raspberry Pi devices in CI.
https://ci.nodejs.org/job/node-test-binary-arm/943/

PR-URL: https://github.com/nodejs/node/pull/5082
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Roman Klauke <romaaan.git@gmail.com>
Ref: https://github.com/nodejs/node/issues/4830
Ref: https://github.com/nodejs/node/issues/3635
Ref: https://github.com/nodejs/node/issues/4526
2016-02-06 08:42:34 -08:00
Rich Trott
c6d29cb81f test: mark test-tick-processor flaky
PR-URL: https://github.com/nodejs/node/pull/4809
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: James M Snell <jasnell@gmail.com>
Ref: https://github.com/nodejs/node/issues/4427
2016-01-24 21:18:08 -08:00
Rich Trott
063c9f6886 test: fix flaky test-cluster-shared-leak
Wait for worker2 to come online before doing anything that might result
in an EPIPE. Fixes flakiness of test on Windows.

Fixes: https://github.com/nodejs/node/issues/3956
PR-URL: https://github.com/nodejs/node/pull/4510
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: James M Snell<jasnell@gmail.com>
2016-01-04 12:03:07 -08:00
Rich Trott
a96d427e08 test: remove flaky designations for tests
Three tests designated as flaky on Linux have not failed on the
continuous integration server in a long time. Removing flaky designation
for these tests.

Fixes: https://github.com/nodejs/node/issues/4446
PR-URL: https://github.com/nodejs/node/pull/4519
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
2016-01-04 09:09:33 -08:00
Brian White
3809126427 test: fix flaky cluster-net-send
Before this commit, it was possible on Windows for the server's
'connection' handler to be called *after* the client socket's
'connect' handler. This caused the 'message' event to be missed
and the test would never end (timing out in CI). This problem
was more easily reproducible on a low resource (slow CPU)
Windows (2012r2) installation.

This commit waits until both handlers have been called before
sending the handle to the master process.

Fixes: https://github.com/nodejs/node/issues/3957
PR-URL: https://github.com/nodejs/node/pull/4444
Reviewed-By: Rich Trott <rtrott@gmail.com>
2015-12-29 00:14:56 -05:00
Brian White
30c0062dd2 test: fix flaky child-process-fork-regr-gh-2847
Windows would die with ECONNRESET most times when running
this particular test. This commit makes handling these errors
more tolerable.

PR-URL: https://github.com/nodejs/node/pull/4442
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
2015-12-28 23:31:32 -05:00
Rich Trott
d09c447ef9 test: mark test-debug-no-context is flaky
test-debug-no-context is flaky on Windows. Mark as such.

Ref: https://github.com/nodejs/node/issues/4343
PR-URL: https://github.com/nodejs/node/pull/4421
Reviewed-By: Brian White <mscdex@mscdex.net>
2015-12-28 08:28:33 -08:00
Rich Trott
afd512253d test: mark test-cluster-shared-leak flaky
test-cluster-shared-leak is flaky on Windows.

Refs: https://github.com/nodejs/node/issues/3956

PR-URL: https://github.com/nodejs/node/pull/4162
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Roman Klauke <romaaan.git@gmail.com>
Reviewed-By: Minwoo Jung <jmwsoft@gmail.com>
2015-12-06 19:18:03 +01:00
Rich Trott
c79b145abc test: fix flaky test-net-socket-local-address
test-net-socket-local-address had a race condition that resulted in
unreliability on FreeBSD and Windows. This changes fixes the issue.

Fixes: https://github.com/nodejs/node/issues/2475
PR-URL: https://github.com/nodejs/node/pull/4109
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
2015-12-03 13:24:44 -08:00
Rich Trott
e25f8683f1 test: mark test flaky on FreeBSD
Mark test-net-socket-local-address flaky on FreeBSD. It times out with
some frequency on CI.

Ref: https://github.com/nodejs/node/issues/2475
PR-URL: https://github.com/nodejs/node/pull/4016
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2015-11-26 19:39:42 -08:00
Rich Trott
16db4c4d5e test: mark cluster-net-send test flaky on windows
See https://github.com/nodejs/node/issues/3957 for details and examples
failures.

Ref: https://github.com/nodejs/node/issues/3957
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
PR-URL: https://github.com/nodejs/node/pull/4006
2015-11-25 09:57:19 -08:00
Rich Trott
fc47e0f8cb test: mark fork regression test flaky on windows
See https://github.com/nodejs/node/issues/3635 for details and failure
examples.

Ref: https://github.com/nodejs/node/issues/3635
PR-URL: https://github.com/nodejs/node/pull/4005
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2015-11-25 09:47:45 -08:00
Rich Trott
b061e3af55 test: skip test if in FreeBSD jail
Test test-net-socket-local-address is flaky in FreeBSD jail but robust
otherwise.

Fixes: https://github.com/nodejs/node/issues/2475
PR-URL: https://github.com/nodejs/node/pull/3995
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2015-11-24 19:38:21 -08:00
Rich Trott
658842280d test: remove flaky status for cluster test
The test did not fail after 9999 runs in continuous integration.

Remove it's flaky status.

Fixes: https://github.com/nodejs/node/issues/2557
PR-URL: https://github.com/nodejs/node/pull/3975
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Roman Klauke <romaaan.git@gmail.com>
2015-11-24 20:14:10 +01:00
Rich Trott
0966ab9996 src: force line buffering for stderr
SmartOS does not line buffer stderr by default, or at least that is the
behavior on the Node project Jenkins server. Force line buffering. This
resolves the flakiness observed on SmartOS for
test-debug-signal-cluster.

PR-URL: https://github.com/nodejs/node/pull/3701
Fixes: https://github.com/nodejs/node/issues/2476
Refs: https://github.com/nodejs/node/pull/3615
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2015-11-10 17:52:44 -08:00
Rich Trott
f6230b9b72 test: remove flaky designation from ls-no-sslv3
This test was marked flaky after failing in CI on arm7-wheezy two months
ago. It has not failed there since. This commit removes the flaky
designation.

Fixes: https://github.com/nodejs/node/issues/2554
PR-URL: https://github.com/nodejs/node/pull/3620
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
2015-11-02 11:46:45 -08:00
Matt Loring
fed0b28754 tools: fix flakiness in test-tick-processor
Per the discussion on #2471, the JS symbols checked for by this test
were occasionally too deep in the stack and were being ignored by the
tick processor.

I have addressed this by increasing the stack depth inspected by the
tick processor and looking for the eval symbol which is more likely
to be present. Additional flakiness was caused by occasional misses
of the code creation event for the JS function being executed. I now
have separate code snippets to test for JS and C++ symbols and if
the code creation event is missed for the JS symbol test then I check
for a percentage of UNKNOWN symbols in processed output. This is
considered a success as the processing scripts in the node repository
are still correctly processing the ticks recieved from the v8
scripts. Further investigation is needed into the v8 profiling
scripts to determine why code creation events are being missed.

PR-URL: https://github.com/nodejs/node/pull/2694
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
2015-09-15 00:12:32 +02:00
Rich Trott
750adddd15 test: unmark test-process-argv-0.js as flaky
https://github.com/nodejs/node/pull/2541 fixed flakiness
in test-process-argv-0.js. However, it was not removed
from the list of flaky tests. This removes it from the
list of flaky tests.

PR-URL: https://github.com/nodejs/node/pull/2613
Reviewed-By: cjihrig - Colin Ihrig <cjihrig@gmail.com>
2015-08-31 21:46:23 -04:00
Alexis Campailla
34e25b2313 test: initial list of flaky tests
This poplulates the lists of flaky tests with tests that failed recently
in Jenkins.

PR-URL: https://github.com/nodejs/node/pull/2424
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
2015-08-27 05:49:55 +02:00
Alexis Campailla
ff05953b8d test: add test configuration templates
Template configuration files for marking tests as flaky.

PR-URL: https://github.com/nodejs/node/pull/2424
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
2015-08-27 05:49:28 +02:00