InlineArray[Valid]/[Invalid] Runtime tests are failing for Mono miniJIT and interpreter

[matouskozak]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=369496

Affected pipelines:

• runtime-extra-platforms

Build error legs:

• Mono android x64 Release @ Ubuntu.1804.Amd64.Android.29.Open

Failing:

• android-x64 Release AllSubsets_Mono_RuntimeTests minijit
• android-x64 Release AllSubsets_Mono_RuntimeTests_Interp monointerpreter

[10:40:25] dbug: Executing command: '/datadisks/disk1/work/9B9908E4/p/microsoft.dotnet.xharness.cli/8.0.0-prerelease.23401.3/runtimes/any/native/adb/linux/adb -s emulator-5556 shell am instrument -e entrypoint:libname InlineArrayValid.dll -w net.dot.Loader_classloader/net.dot.MonoRunner'
        [10:40:29] info: Running instrumentation class net.dot.MonoRunner took 4.5776835 seconds
        [10:40:29] dbug: Exit code: 0
                         Std out:
                         INSTRUMENTATION_RESULT: return-code=101
                         INSTRUMENTATION_CODE: 101
                         
                         
                         
        [10:40:30] info: Instrumentation finished normally with exit code 101
        [10:40:33] dbug: Executing command: '/datadisks/disk1/work/9B9908E4/p/microsoft.dotnet.xharness.cli/8.0.0-prerelease.23401.3/runtimes/any/native/adb/linux/adb -s emulator-5556 logcat -d '
        [10:40:33] info: Wrote current ADB log to /datadisks/disk1/work/9B9908E4/w/AB97096C/uploads/Reports/Loader.classloader/InlineArray/InlineArrayValid/adb-logcat-net.dot.Loader_classloader-net.dot.MonoRunner.log
        [10:40:33] fail: Non-success instrumentation exit code: 101, expected: 100
        [10:40:33] dbug: Saving diagnostics data to '/datadisks/disk1/work/9B9908E4/w/AB97096C/e/diagnostics.json'
        XHarness exit code: 1 (TESTS_FAILED)

[10:39:01] dbug: Executing command: '/datadisks/disk1/work/9B9908E4/p/microsoft.dotnet.xharness.cli/8.0.0-prerelease.23401.3/runtimes/any/native/adb/linux/adb -s emulator-5556 shell am instrument -e entrypoint:libname InlineArrayInvalid.dll -w net.dot.Loader_classloader/net.dot.MonoRunner'
        [10:39:05] info: Running instrumentation class net.dot.MonoRunner took 3.6335327 seconds
        [10:39:05] dbug: Exit code: 0
                         Std out:
                         INSTRUMENTATION_RESULT: shortMsg=Process crashed.
                         INSTRUMENTATION_CODE: 0
                         
                         
                         
        [10:39:05] info: Short message:
                         Process crashed.
        [10:39:05] fail: No value for 'return-code' provided in instrumentation result. This may indicate a crashed test (see log)

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorPattern": "Loader\\/classloader\\/InlineArray\\/[\\w\\/.\\s]+\\[FAIL\\]",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=369496
Error message validated: Loader\/classloader\/InlineArray\/[\w\/.\s]+\[FAIL\]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 8/14/2023 10:41:08 AM UTC

Report

Build	Definition	Test	Pull Request
374175	dotnet/runtime	Loader.classloader.WorkItemExecution	#90583
373829	dotnet/runtime	Loader/classloader/regressions/GitHub_82187/GitHub_82187/GitHub_82187.sh	#90412
372849	dotnet/runtime	Loader.classloader.WorkItemExecution	#90519
372550	dotnet/runtime	Loader.classloader.WorkItemExecution
372445	dotnet/runtime	Loader.classloader.WorkItemExecution	#90270
372307	dotnet/runtime	Loader.classloader.WorkItemExecution
372186	dotnet/runtime	Loader.classloader.WorkItemExecution
371961	dotnet/runtime	Loader.classloader.WorkItemExecution	#90023
369496	dotnet/runtime	Loader.classloader.WorkItemExecution

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
2	9	9

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Error Blob

{
  "ErrorMessage": "Non-success instrumentation exit code: 101, expected: 100",
  "ErrorMessage": "No value for 'return-code' provided in instrumentation result. This may indicate a crashed test (see log)",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": true
}

[10:40:25] dbug: Executing command: '/datadisks/disk1/work/9B9908E4/p/microsoft.dotnet.xharness.cli/8.0.0-prerelease.23401.3/runtimes/any/native/adb/linux/adb -s emulator-5556 shell am instrument -e entrypoint:libname InlineArrayValid.dll -w net.dot.Loader_classloader/net.dot.MonoRunner'
        [10:40:29] info: Running instrumentation class net.dot.MonoRunner took 4.5776835 seconds
        [10:40:29] dbug: Exit code: 0
                         Std out:
                         INSTRUMENTATION_RESULT: return-code=101
                         INSTRUMENTATION_CODE: 101
                         
                         
                         
        [10:40:30] info: Instrumentation finished normally with exit code 101
        [10:40:33] dbug: Executing command: '/datadisks/disk1/work/9B9908E4/p/microsoft.dotnet.xharness.cli/8.0.0-prerelease.23401.3/runtimes/any/native/adb/linux/adb -s emulator-5556 logcat -d '
        [10:40:33] info: Wrote current ADB log to /datadisks/disk1/work/9B9908E4/w/AB97096C/uploads/Reports/Loader.classloader/InlineArray/InlineArrayValid/adb-logcat-net.dot.Loader_classloader-net.dot.MonoRunner.log
        [10:40:33] fail: Non-success instrumentation exit code: 101, expected: 100
        [10:40:33] dbug: Saving diagnostics data to '/datadisks/disk1/work/9B9908E4/w/AB97096C/e/diagnostics.json'
        XHarness exit code: 1 (TESTS_FAILED)

[10:39:01] dbug: Executing command: '/datadisks/disk1/work/9B9908E4/p/microsoft.dotnet.xharness.cli/8.0.0-prerelease.23401.3/runtimes/any/native/adb/linux/adb -s emulator-5556 shell am instrument -e entrypoint:libname InlineArrayInvalid.dll -w net.dot.Loader_classloader/net.dot.MonoRunner'
        [10:39:05] info: Running instrumentation class net.dot.MonoRunner took 3.6335327 seconds
        [10:39:05] dbug: Exit code: 0
                         Std out:
                         INSTRUMENTATION_RESULT: shortMsg=Process crashed.
                         INSTRUMENTATION_CODE: 0
                         
                         
                         
        [10:39:05] info: Short message:
                         Process crashed.
        [10:39:05] fail: No value for 'return-code' provided in instrumentation result. This may indicate a crashed test (see log)

Reproduction Steps

android-x64 Release AllSubsets_Mono_RuntimeTests_[minijit]/[Interp monointerpreter] CI pipelines are failing, e.g.: https://dev.azure.com/dnceng-public/public/_build/results?buildId=369496&view=logs&jobId=e847dbc9-1eac-5acf-c55e-fa567d2d051e&j=353cbaf6-57aa-5b05-dc8e-ce12fdf7e230&t=a84f0057-2a70-587b-3a48-3788b239fbbb

Author:	matouskozak
Assignees:	-
Labels:	blocking-clean-ci, area-Infrastructure-mono, Known Build Error
Milestone:	-

[matouskozak]

The tests were enabled in #90192 and since then started failing on runtime-extra-platforms.

[SamMonoRT]

@AaronRobinsonMSFT - If these newly added tests can't be fixed for Mono (which at the moment we don't have cycles for RC1 cutoff) I suggest we disable the failing tests on mono lanes to get extra-platforms JIT/Interpreter lanes running again. The extra-platforms is not green and we are trying to make sure it doesn't regress further.

cc @lambdageek

[AaronRobinsonMSFT]

@AaronRobinsonMSFT - If these newly added tests can't be fixed for Mono (which at the moment we don't have cycles for RC1 cutoff) I suggest we disable the failing tests on mono lanes to get extra-platforms JIT/Interpreter lanes running again.

Sure. I encourage the mono owners to handle that specifically. The disable mechanisms for some of these mono scenrios are unknown to a lot of us. I'm open to reviewing any PR, but simply disabling a test should be relatively simple to handle, no?

[SamMonoRT]

@AaronRobinsonMSFT - If these newly added tests can't be fixed for Mono (which at the moment we don't have cycles for RC1 cutoff) I suggest we disable the failing tests on mono lanes to get extra-platforms JIT/Interpreter lanes running again.

Sure. I encourage the mono owners to handle that specifically. The disable mechanisms for some of these mono scenrios are unknown to a lot of us. I'm open to reviewing any PR, but simply disabling a test should be relatively simple to handle, no?

Sounds good - @matouskozak will create a PR to disable those and add you for a quick review.

[lambdageek]

The tests are working on desktop, right? it's just android-x64 that's busted?

[lambdageek]

Looking at the CI run for #90192 - it doesn't look like the runtime tests ran on Mono at all??? @steveisok @directhex Are the CI triggers not running tests with Mono if only src/tests changes???

Nevermind. the lanes are just named differently. For example these ran runtime tests:

Build osx-x64 Release AllSubsets_Mono_Interpreter_RuntimeTests monointerpreter
Build osx-x64 Release AllSubsets_Mono_Minijit_RuntimeTests minijit

So it is just mobile that is acting up

[SamMonoRT]

Looking at the CI run for #90192 - it doesn't look like the runtime tests ran on Mono at all??? @steveisok @directhex Are the CI triggers not running tests with Mono if only src/tests changes???

It's the lanes in extra-platforms, which most devs outside mono don't run

[Sam edit] - Interesting, yeah, seems like no Mono runtime tests lanes were triggered at all. But the failures have been in Android only lanes -- [Sam Edit #2] - Some mono lanes have indeed run. Only failures are in extra-platforms.