Investigate flaky parallel/test-performance

[apapirovski]

The recently changed parallel/test-performance appears to unfortunately be flaky now. Recently failed on aix61-ppc64. More info:

assert.js:249
    throw err;
    ^

AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert(Math.abs(delta) < 1000)

    at checkNodeTiming (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test/parallel/test-performance.js:118:7)
    at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test/parallel/test-performance.js:125:1)
    at Module._compile (module.js:670:30)
    at Object.Module._extensions..js (module.js:681:10)
    at Module.load (module.js:581:32)
    at tryModuleLoad (module.js:521:12)
    at Function.Module._load (module.js:513:3)
    at Function.Module.runMain (module.js:711:10)
    at startup (bootstrap_node.js:223:16)
    at bootstrapNodeJSCore (bootstrap_node.js:562:3)

/cc @TimothyGu

[apapirovski]

Failed again recently: https://ci.nodejs.org/job/node-test-commit-linux-containered/2768/nodes=ubuntu1604_sharedlibs_debug_x64/tapResults/

[Trott]

The timing stuff sure suggests to me that the test ought to be in sequential rather than parallel.

[Trott]

Oh, yes, definitely can fail due to system load.

$ tools/test.py --repeat 192 -j 96 test/parallel/test-performance.js 
=== release test-performance ===                    
Path: parallel/test-performance
assert.js:249
    throw err;
    ^

AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert(Math.abs(delta) < 1000)

    at checkNodeTiming (/Users/trott/io.js/test/parallel/test-performance.js:118:7)
    at Object.<anonymous> (/Users/trott/io.js/test/parallel/test-performance.js:125:1)
    at Module._compile (module.js:670:30)
    at Object.Module._extensions..js (module.js:681:10)
    at Module.load (module.js:581:32)
    at tryModuleLoad (module.js:521:12)
    at Function.Module._load (module.js:513:3)
    at Function.Module.runMain (module.js:711:10)
    at startup (bootstrap_node.js:223:16)
    at bootstrapNodeJSCore (bootstrap_node.js:562:3)
Command: out/Release/node /Users/trott/io.js/test/parallel/test-performance.js
=== release test-performance ===                    
Path: parallel/test-performance
assert.js:249
    throw err;
    ^

AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert(Math.abs(delta) < 1000)

    at checkNodeTiming (/Users/trott/io.js/test/parallel/test-performance.js:118:7)
    at Object.<anonymous> (/Users/trott/io.js/test/parallel/test-performance.js:125:1)
    at Module._compile (module.js:670:30)
    at Object.Module._extensions..js (module.js:681:10)
    at Module.load (module.js:581:32)
    at tryModuleLoad (module.js:521:12)
    at Function.Module._load (module.js:513:3)
    at Function.Module.runMain (module.js:711:10)
    at startup (bootstrap_node.js:223:16)
    at bootstrapNodeJSCore (bootstrap_node.js:562:3)
Command: out/Release/node /Users/trott/io.js/test/parallel/test-performance.js
=== release test-performance ===                    
Path: parallel/test-performance
assert.js:249
    throw err;
    ^

AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert(Math.abs(delta) < 1000)

    at checkNodeTiming (/Users/trott/io.js/test/parallel/test-performance.js:118:7)
    at Object.<anonymous> (/Users/trott/io.js/test/parallel/test-performance.js:125:1)
    at Module._compile (module.js:670:30)
    at Object.Module._extensions..js (module.js:681:10)
    at Module.load (module.js:581:32)
    at tryModuleLoad (module.js:521:12)
    at Function.Module._load (module.js:513:3)
    at Function.Module.runMain (module.js:711:10)
    at startup (bootstrap_node.js:223:16)
    at bootstrapNodeJSCore (bootstrap_node.js:562:3)
Command: out/Release/node /Users/trott/io.js/test/parallel/test-performance.js
[00:11|% 100|+ 189|-   3]: Done                     
$

[Trott]

PR to address this: #19228

[joyeecheung]

This happened again in sequential: nodejs/reliability#14

Reason	sequential/test-performance
Type	JS_TEST_FAILURE
Failed PR	2 (#22318, #22288)
Appeared	test-digitalocean-ubuntu1604_sharedlibs_container-x64-7, test-requireio_andineck-debian9-armv6l_pi1p-1
First CI	https://ci.nodejs.org/job/node-test-pull-request/16459/
Last CI	https://ci.nodejs.org/job/node-test-pull-request/16462/

Example

not ok 2289 sequential/test-performance
  ---
  duration_ms: 4.558
  severity: fail
  exitcode: 1
  stack: |-
    assert.js:338
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
    
      assert(Math.abs(delta) < 1000)
    
        at checkNodeTiming (/home/iojs/build/workspace/node-test-commit-linux-containered/nodes/ubuntu1604_sharedlibs_debug_x64/test/sequential/test-performance.js:64:7)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/nodes/ubuntu1604_sharedlibs_debug_x64/test/sequential/test-performance.js:73:1)
        at Module._compile (internal/modules/cjs/loader.js:689:30)
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
        at Module.load (internal/modules/cjs/loader.js:599:32)
        at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
        at Function.Module._load (internal/modules/cjs/loader.js:530:3)
        at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
        at startup (internal/bootstrap/node.js:266:19)
        at bootstrapNodeJSCore (internal/bootstrap/node.js:611:3)
  ...