Test failures on macOS Sierra 10.12 (GM) with Xcode 8.0 (GM)

[dnalborczyk]

• Version: master (latest)
• Platform: macOS Sierra 10.12 (GM), Xcode 8.0 (GM)
• Subsystem: -

I don't know who is responsible for the failures, macOS, Xcode or a combination.

./configure
make test

[==========] Running 22 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 4 tests from UtilTest
[ RUN      ] UtilTest.ListHead
[       OK ] UtilTest.ListHead (0 ms)
[ RUN      ] UtilTest.StringEqualNoCase
[       OK ] UtilTest.StringEqualNoCase (0 ms)
[ RUN      ] UtilTest.StringEqualNoCaseN
[       OK ] UtilTest.StringEqualNoCaseN (0 ms)
[ RUN      ] UtilTest.ToLower
[       OK ] UtilTest.ToLower (0 ms)
[----------] 4 tests from UtilTest (0 ms total)

[----------] 18 tests from InspectorSocketTest
[ RUN      ] InspectorSocketTest.ReadsAndWritesInspectorMessage
[       OK ] InspectorSocketTest.ReadsAndWritesInspectorMessage (0 ms)
[ RUN      ] InspectorSocketTest.BufferEdgeCases
[       OK ] InspectorSocketTest.BufferEdgeCases (1 ms)
[ RUN      ] InspectorSocketTest.AcceptsRequestInSeveralWrites
[       OK ] InspectorSocketTest.AcceptsRequestInSeveralWrites (0 ms)
[ RUN      ] InspectorSocketTest.ExtraTextBeforeRequest
../test/cctest/test_inspector_socket.cc:485: Failure
Value of: 0
Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
Which is: 1
[  FAILED  ] InspectorSocketTest.ExtraTextBeforeRequest (0 ms)
[ RUN      ] InspectorSocketTest.ExtraLettersBeforeRequest
../test/cctest/test_inspector_socket.cc:497: Failure
Value of: 0
Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
Which is: 1
[  FAILED  ] InspectorSocketTest.ExtraLettersBeforeRequest (1 ms)
[ RUN      ] InspectorSocketTest.RequestWithoutKey
../test/cctest/test_inspector_socket.cc:512: Failure
Value of: 0
Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
Which is: 1
[  FAILED  ] InspectorSocketTest.RequestWithoutKey (0 ms)
[ RUN      ] InspectorSocketTest.KillsConnectionOnProtocolViolation
[       OK ] InspectorSocketTest.KillsConnectionOnProtocolViolation (0 ms)
[ RUN      ] InspectorSocketTest.CanStopReadingFromInspector
[       OK ] InspectorSocketTest.CanStopReadingFromInspector (0 ms)
[ RUN      ] InspectorSocketTest.CloseDoesNotNotifyReadCallback
[       OK ] InspectorSocketTest.CloseDoesNotNotifyReadCallback (1 ms)
[ RUN      ] InspectorSocketTest.CloseWorksWithoutReadEnabled
[       OK ] InspectorSocketTest.CloseWorksWithoutReadEnabled (0 ms)
[ RUN      ] InspectorSocketTest.ReportsHttpGet
[       OK ] InspectorSocketTest.ReportsHttpGet (4 ms)
[ RUN      ] InspectorSocketTest.HandshakeCanBeCanceled
[       OK ] InspectorSocketTest.HandshakeCanBeCanceled (1 ms)
[ RUN      ] InspectorSocketTest.GetThenHandshake
[       OK ] InspectorSocketTest.GetThenHandshake (0 ms)
[ RUN      ] InspectorSocketTest.WriteBeforeHandshake
[       OK ] InspectorSocketTest.WriteBeforeHandshake (0 ms)
[ RUN      ] InspectorSocketTest.CleanupSocketAfterEOF
[       OK ] InspectorSocketTest.CleanupSocketAfterEOF (4 ms)
[ RUN      ] InspectorSocketTest.EOFBeforeHandshake
[       OK ] InspectorSocketTest.EOFBeforeHandshake (1 ms)
[ RUN      ] InspectorSocketTest.Send1Mb
[       OK ] InspectorSocketTest.Send1Mb (15 ms)
[ RUN      ] InspectorSocketTest.ErrorCleansUpTheSocket
[       OK ] InspectorSocketTest.ErrorCleansUpTheSocket (1 ms)
[----------] 18 tests from InspectorSocketTest (29 ms total)

[----------] Global test environment tear-down
[==========] 22 tests from 2 test cases ran. (29 ms total)
[  PASSED  ] 19 tests.
[  FAILED  ] 3 tests, listed below:
[  FAILED  ] InspectorSocketTest.ExtraTextBeforeRequest
[  FAILED  ] InspectorSocketTest.ExtraLettersBeforeRequest
[  FAILED  ] InspectorSocketTest.RequestWithoutKey

 3 FAILED TESTS
make[1]: *** [cctest] Error 1
make: *** [test] Error 2

[bnoordhuis]

Those are the sames failures as #8034 (comment). For some reason it happens on some systems, but not others.

[eugeneo]

I will take a look. I believe this is a race condition, so far I did not see it on any system I use...

[eugeneo]

Please try if #8505 fixes this.

[dnalborczyk]

unfortunately same failures, with different output.
the failing test cases also seem to "pause" for a bit while running. not sure if that is intended.

[==========] Running 22 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 4 tests from UtilTest
[ RUN      ] UtilTest.ListHead
[       OK ] UtilTest.ListHead (0 ms)
[ RUN      ] UtilTest.StringEqualNoCase
[       OK ] UtilTest.StringEqualNoCase (0 ms)
[ RUN      ] UtilTest.StringEqualNoCaseN
[       OK ] UtilTest.StringEqualNoCaseN (0 ms)
[ RUN      ] UtilTest.ToLower
[       OK ] UtilTest.ToLower (0 ms)
[----------] 4 tests from UtilTest (0 ms total)

[----------] 18 tests from InspectorSocketTest
[ RUN      ] InspectorSocketTest.ReadsAndWritesInspectorMessage
[       OK ] InspectorSocketTest.ReadsAndWritesInspectorMessage (1 ms)
[ RUN      ] InspectorSocketTest.BufferEdgeCases
[       OK ] InspectorSocketTest.BufferEdgeCases (1 ms)
[ RUN      ] InspectorSocketTest.AcceptsRequestInSeveralWrites
[       OK ] InspectorSocketTest.AcceptsRequestInSeveralWrites (0 ms)
[ RUN      ] InspectorSocketTest.ExtraTextBeforeRequest
../test/cctest/test_inspector_socket.cc:311: Failure
Value of: (uv_is_active(reinterpret_cast<uv_handle_t*>(&socket)))
  Actual: true
Expected: false
[  FAILED  ] InspectorSocketTest.ExtraTextBeforeRequest (4999 ms)
[ RUN      ] InspectorSocketTest.ExtraLettersBeforeRequest
../test/cctest/test_inspector_socket.cc:311: Failure
Value of: (uv_is_active(reinterpret_cast<uv_handle_t*>(&socket)))
  Actual: true
Expected: false
[  FAILED  ] InspectorSocketTest.ExtraLettersBeforeRequest (5000 ms)
[ RUN      ] InspectorSocketTest.RequestWithoutKey
../test/cctest/test_inspector_socket.cc:311: Failure
Value of: (uv_is_active(reinterpret_cast<uv_handle_t*>(&socket)))
  Actual: true
Expected: false
[  FAILED  ] InspectorSocketTest.RequestWithoutKey (5000 ms)
[ RUN      ] InspectorSocketTest.KillsConnectionOnProtocolViolation
[       OK ] InspectorSocketTest.KillsConnectionOnProtocolViolation (0 ms)
[ RUN      ] InspectorSocketTest.CanStopReadingFromInspector
[       OK ] InspectorSocketTest.CanStopReadingFromInspector (1 ms)
[ RUN      ] InspectorSocketTest.CloseDoesNotNotifyReadCallback
[       OK ] InspectorSocketTest.CloseDoesNotNotifyReadCallback (0 ms)
[ RUN      ] InspectorSocketTest.CloseWorksWithoutReadEnabled
[       OK ] InspectorSocketTest.CloseWorksWithoutReadEnabled (0 ms)
[ RUN      ] InspectorSocketTest.ReportsHttpGet
[       OK ] InspectorSocketTest.ReportsHttpGet (5 ms)
[ RUN      ] InspectorSocketTest.HandshakeCanBeCanceled
[       OK ] InspectorSocketTest.HandshakeCanBeCanceled (0 ms)
[ RUN      ] InspectorSocketTest.GetThenHandshake
[       OK ] InspectorSocketTest.GetThenHandshake (1 ms)
[ RUN      ] InspectorSocketTest.WriteBeforeHandshake
[       OK ] InspectorSocketTest.WriteBeforeHandshake (0 ms)
[ RUN      ] InspectorSocketTest.CleanupSocketAfterEOF
[       OK ] InspectorSocketTest.CleanupSocketAfterEOF (4 ms)
[ RUN      ] InspectorSocketTest.EOFBeforeHandshake
[       OK ] InspectorSocketTest.EOFBeforeHandshake (0 ms)
[ RUN      ] InspectorSocketTest.Send1Mb
[       OK ] InspectorSocketTest.Send1Mb (15 ms)
[ RUN      ] InspectorSocketTest.ErrorCleansUpTheSocket
[       OK ] InspectorSocketTest.ErrorCleansUpTheSocket (0 ms)
[----------] 18 tests from InspectorSocketTest (15027 ms total)

[----------] Global test environment tear-down
[==========] 22 tests from 2 test cases ran. (15027 ms total)
[  PASSED  ] 19 tests.
[  FAILED  ] 3 tests, listed below:
[  FAILED  ] InspectorSocketTest.ExtraTextBeforeRequest
[  FAILED  ] InspectorSocketTest.ExtraLettersBeforeRequest
[  FAILED  ] InspectorSocketTest.RequestWithoutKey

 3 FAILED TESTS
make[1]: *** [cctest] Error 1
make: *** [test] Error 2

[eugeneo]

Please try the newer revisions. I just realized there was a coding mistake and socket is actually not a variable but a function from the platform library...

[dnalborczyk]

👍 looks good!!

[==========] Running 22 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 4 tests from UtilTest
[ RUN      ] UtilTest.ListHead
[       OK ] UtilTest.ListHead (0 ms)
[ RUN      ] UtilTest.StringEqualNoCase
[       OK ] UtilTest.StringEqualNoCase (0 ms)
[ RUN      ] UtilTest.StringEqualNoCaseN
[       OK ] UtilTest.StringEqualNoCaseN (0 ms)
[ RUN      ] UtilTest.ToLower
[       OK ] UtilTest.ToLower (0 ms)
[----------] 4 tests from UtilTest (0 ms total)

[----------] 18 tests from InspectorSocketTest
[ RUN      ] InspectorSocketTest.ReadsAndWritesInspectorMessage
[       OK ] InspectorSocketTest.ReadsAndWritesInspectorMessage (1 ms)
[ RUN      ] InspectorSocketTest.BufferEdgeCases
[       OK ] InspectorSocketTest.BufferEdgeCases (0 ms)
[ RUN      ] InspectorSocketTest.AcceptsRequestInSeveralWrites
[       OK ] InspectorSocketTest.AcceptsRequestInSeveralWrites (1 ms)
[ RUN      ] InspectorSocketTest.ExtraTextBeforeRequest
[       OK ] InspectorSocketTest.ExtraTextBeforeRequest (0 ms)
[ RUN      ] InspectorSocketTest.ExtraLettersBeforeRequest
[       OK ] InspectorSocketTest.ExtraLettersBeforeRequest (0 ms)
[ RUN      ] InspectorSocketTest.RequestWithoutKey
[       OK ] InspectorSocketTest.RequestWithoutKey (0 ms)
[ RUN      ] InspectorSocketTest.KillsConnectionOnProtocolViolation
[       OK ] InspectorSocketTest.KillsConnectionOnProtocolViolation (1 ms)
[ RUN      ] InspectorSocketTest.CanStopReadingFromInspector
[       OK ] InspectorSocketTest.CanStopReadingFromInspector (0 ms)
[ RUN      ] InspectorSocketTest.CloseDoesNotNotifyReadCallback
[       OK ] InspectorSocketTest.CloseDoesNotNotifyReadCallback (0 ms)
[ RUN      ] InspectorSocketTest.CloseWorksWithoutReadEnabled
[       OK ] InspectorSocketTest.CloseWorksWithoutReadEnabled (0 ms)
[ RUN      ] InspectorSocketTest.ReportsHttpGet
[       OK ] InspectorSocketTest.ReportsHttpGet (5 ms)
[ RUN      ] InspectorSocketTest.HandshakeCanBeCanceled
[       OK ] InspectorSocketTest.HandshakeCanBeCanceled (0 ms)
[ RUN      ] InspectorSocketTest.GetThenHandshake
[       OK ] InspectorSocketTest.GetThenHandshake (0 ms)
[ RUN      ] InspectorSocketTest.WriteBeforeHandshake
[       OK ] InspectorSocketTest.WriteBeforeHandshake (0 ms)
[ RUN      ] InspectorSocketTest.CleanupSocketAfterEOF
[       OK ] InspectorSocketTest.CleanupSocketAfterEOF (4 ms)
[ RUN      ] InspectorSocketTest.EOFBeforeHandshake
[       OK ] InspectorSocketTest.EOFBeforeHandshake (1 ms)
[ RUN      ] InspectorSocketTest.Send1Mb
[       OK ] InspectorSocketTest.Send1Mb (14 ms)
[ RUN      ] InspectorSocketTest.ErrorCleansUpTheSocket
[       OK ] InspectorSocketTest.ErrorCleansUpTheSocket (0 ms)
[----------] 18 tests from InspectorSocketTest (27 ms total)

[----------] Global test environment tear-down
[==========] 22 tests from 2 test cases ran. (27 ms total)
[  PASSED  ] 22 tests.
/usr/bin/python tools/test.py --mode=release -J \
        addons doctool known_issues message pseudo-tty parallel sequential
[01:16|% 100|+ 1234|-   0]: Done
/Applications/Xcode.app/Contents/Developer/usr/bin/make lint
./node tools/eslint/bin/eslint.js --cache --rulesdir=tools/eslint-rules \
      benchmark lib test tools
Total errors found: 0

[Thylossus]

Hi,

I would like to add that I'm experiencing the same issue. I forked and cloned the repo just this morning and I am using Windows. So this issue is probably not Mac specific.

Specs:

• Version: master (latest)
• Platform: Windows 8.1 (Version 6.3 Build 9600) with Visual Studio 2015

Running main() from gtest_main.cc
[==========] Running 22 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 4 tests from UtilTest
[ RUN      ] UtilTest.ListHead
[       OK ] UtilTest.ListHead (1 ms)
[ RUN      ] UtilTest.StringEqualNoCase
[       OK ] UtilTest.StringEqualNoCase (0 ms)
[ RUN      ] UtilTest.StringEqualNoCaseN
[       OK ] UtilTest.StringEqualNoCaseN (0 ms)
[ RUN      ] UtilTest.ToLower
[       OK ] UtilTest.ToLower (0 ms)
[----------] 4 tests from UtilTest (2 ms total)

[----------] 18 tests from InspectorSocketTest
[ RUN      ] InspectorSocketTest.ReadsAndWritesInspectorMessage
[       OK ] InspectorSocketTest.ReadsAndWritesInspectorMessage (13 ms)
[ RUN      ] InspectorSocketTest.BufferEdgeCases
[       OK ] InspectorSocketTest.BufferEdgeCases (4 ms)
[ RUN      ] InspectorSocketTest.AcceptsRequestInSeveralWrites
[       OK ] InspectorSocketTest.AcceptsRequestInSeveralWrites (4 ms)
[ RUN      ] InspectorSocketTest.ExtraTextBeforeRequest
test\cctest\test_inspector_socket.cc(505): error: Value of: 0
Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
Which is: 1
[  FAILED  ] InspectorSocketTest.ExtraTextBeforeRequest (5 ms)
[ RUN      ] InspectorSocketTest.ExtraLettersBeforeRequest
test\cctest\test_inspector_socket.cc(517): error: Value of: 0
Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
Which is: 1
[  FAILED  ] InspectorSocketTest.ExtraLettersBeforeRequest (4 ms)
[ RUN      ] InspectorSocketTest.RequestWithoutKey
test\cctest\test_inspector_socket.cc(532): error: Value of: 0
Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
Which is: 1
[  FAILED  ] InspectorSocketTest.RequestWithoutKey (5 ms)
[ RUN      ] InspectorSocketTest.KillsConnectionOnProtocolViolation
[       OK ] InspectorSocketTest.KillsConnectionOnProtocolViolation (5 ms)
[ RUN      ] InspectorSocketTest.CanStopReadingFromInspector
[       OK ] InspectorSocketTest.CanStopReadingFromInspector (5 ms)
[ RUN      ] InspectorSocketTest.CloseDoesNotNotifyReadCallback
[       OK ] InspectorSocketTest.CloseDoesNotNotifyReadCallback (6 ms)
[ RUN      ] InspectorSocketTest.CloseWorksWithoutReadEnabled
[       OK ] InspectorSocketTest.CloseWorksWithoutReadEnabled (6 ms)
[ RUN      ] InspectorSocketTest.ReportsHttpGet
[       OK ] InspectorSocketTest.ReportsHttpGet (25 ms)
[ RUN      ] InspectorSocketTest.HandshakeCanBeCanceled
[       OK ] InspectorSocketTest.HandshakeCanBeCanceled (6 ms)
[ RUN      ] InspectorSocketTest.GetThenHandshake
[       OK ] InspectorSocketTest.GetThenHandshake (6 ms)
[ RUN      ] InspectorSocketTest.WriteBeforeHandshake
[       OK ] InspectorSocketTest.WriteBeforeHandshake (5 ms)
[ RUN      ] InspectorSocketTest.CleanupSocketAfterEOF
[       OK ] InspectorSocketTest.CleanupSocketAfterEOF (23 ms)
[ RUN      ] InspectorSocketTest.EOFBeforeHandshake
[       OK ] InspectorSocketTest.EOFBeforeHandshake (5 ms)
[ RUN      ] InspectorSocketTest.Send1Mb
[       OK ] InspectorSocketTest.Send1Mb (43 ms)
[ RUN      ] InspectorSocketTest.ErrorCleansUpTheSocket
[       OK ] InspectorSocketTest.ErrorCleansUpTheSocket (5 ms)
[----------] 18 tests from InspectorSocketTest (184 ms total)

[----------] Global test environment tear-down
[==========] 22 tests from 2 test cases ran. (188 ms total)
[  PASSED  ] 19 tests.
[  FAILED  ] 3 tests, listed below:
[  FAILED  ] InspectorSocketTest.ExtraTextBeforeRequest
[  FAILED  ] InspectorSocketTest.ExtraLettersBeforeRequest
[  FAILED  ] InspectorSocketTest.RequestWithoutKey

 3 FAILED TESTS