Half.TryParse assertion failure on System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar

[jozkee]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=344043
Build error leg or test failing: System.Text.Json.Tests.WorkItemExecution
Pull request: #89036

Error Blob

{
  "ErrorMessage": "System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Reproduction Steps

While adding support for Half to System.Text.Json (#88962), I noticed one test triggers the assertion failure consistently on macOS-12.4.
Note: This didn't repro for me locally on macOS 13.4.1 arm64.

The test attempts to deserialize floting-point literals (NaN, infinity, -Infinity) in multiple casings/variants that are invalid for S.T.Json. Error happens on line 883.

runtime/src/libraries/System.Text.Json/tests/Common/NumberHandlingTests.cs

Lines 848 to 883 in 43fe8a8

	[Theory]
	[InlineData("naN")]
	[InlineData("Nan")]
	[InlineData("NAN")]
	[InlineData("+Infinity")]
	[InlineData("+infinity")]
	[InlineData("infinity")]
	[InlineData("infinitY")]
	[InlineData("INFINITY")]
	[InlineData("+INFINITY")]
	[InlineData("-infinity")]
	[InlineData("-infinitY")]
	[InlineData("-INFINITY")]
	[InlineData(" NaN")]
	[InlineData("NaN ")]
	[InlineData(" Infinity")]
	[InlineData(" -Infinity")]
	[InlineData("Infinity ")]
	[InlineData("-Infinity ")]
	[InlineData("a-Infinity")]
	[InlineData("NaNa")]
	[InlineData("Infinitya")]
	[InlineData("-Infinitya")]
	#pragma warning disable xUnit1025 // Theory method 'FloatingPointConstants_Fail' on test class 'NumberHandlingTests' has InlineData duplicate(s)
	[InlineData("\u006EaN")] // "naN"
	[InlineData("\u0020Inf\u0069ni\u0074y")] // " Infinity"
	[InlineData("\u002BInf\u0069nity")] // "+Infinity"
	#pragma warning restore xUnit1025
	public async Task FloatingPointConstants_Fail(string testString)
	{
	string testStringAsJson = $@"""{testString}""";
	string testJson;
	#if NETCOREAPP
	testJson = @$"{{""HalfNumber"":{testStringAsJson}}}";
	await Assert.ThrowsAsync<JsonException>(async () => await Serializer.DeserializeWrapper<StructWithNumbers>(testJson, s_optionsAllowFloatConstants));
	await Assert.ThrowsAsync<JsonException>(async () => await Serializer.DeserializeWrapper<StructWithNumbers>(testJson, s_optionReadFromStr));

Helix log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-88962-merge-9757e201f6374e20b5/System.Text.Json.Tests/1/console.78feb75a.log?helixlogtype=result

Call stack:

/private/tmp/helix/working/9EC208C7/w/A9F108F7/e /private/tmp/helix/working/9EC208C7/w/A9F108F7/e
  Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.Json.Tests (found 7160 of 7231 test cases)
  Starting:    System.Text.Json.Tests (parallel test collections = on, max threads = 6)
Process terminated. Assertion failed.
   at System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar(Byte& charA, Int32 lengthA, Byte& charB, Int32 lengthB) in /_/src/libraries/System.Private.CoreLib/src/System/Globalization/Ordinal.Utf8.cs:line 308
   at System.Number.TryParseFloat[TChar,TFloat](ReadOnlySpan`1 value, NumberStyles styles, NumberFormatInfo info, TFloat& result) in /_/src/libraries/System.Private.CoreLib/src/System/Number.Parsing.cs:line 1229
   at System.Half.TryParse(ReadOnlySpan`1 utf8Text, NumberStyles style, IFormatProvider provider, Half& result) in /_/src/libraries/System.Private.CoreLib/src/System/Half.cs:line 2238
   at System.Text.Json.Serialization.Converters.HalfConverter.TryParse(ReadOnlySpan`1 buffer, Half& result) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 197
   at System.Text.Json.Serialization.Converters.HalfConverter.ReadCore(Utf8JsonReader& reader) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 50
   at System.Text.Json.Serialization.Converters.HalfConverter.ReadNumberWithCustomHandling(Utf8JsonReader& reader, JsonNumberHandling handling, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 115
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs:line 193
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonPropertyInfoOfT.cs:line 308
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Object/ObjectDefaultConverter.cs:line 49
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs:line 258
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.ReadCore.cs:line 51
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.ReadHelper.cs:line 22
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.Span.cs:line 160
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs:line 443
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs:line 55
   at System.Text.Json.Serialization.Tests.JsonSerializerWrapper.StringSerializerWrapper.DeserializeWrapper[T](String json, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/JsonSerializerWrapper.Reflection.cs:line 129
   at System.Text.Json.Serialization.Tests.NumberHandlingTests.<>c__DisplayClass39_0.<<FloatingPointConstants_Fail>b__1>d.MoveNext() in /_/src/libraries/System.Text.Json/tests/Common/NumberHandlingTests.cs:line 883
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) in /_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs:line 38
   at System.Text.Json.Serialization.Tests.NumberHandlingTests.<>c__DisplayClass39_0.<FloatingPointConstants_Fail>b__1()

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=344043
Error message validated: System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/18/2023 11:30:55 PM UTC

Report

Build	Definition	Test	Pull Request
344444	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89153
344421	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89154
344427	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89155
344398	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89151
344340	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#87438
344330	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89061
344321	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89147
344317	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89123
344290	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#87847
344296	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89102
344284	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89144
344268	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89130
344261	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#85328
344239	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#86089
344207	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89129
344182	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89135
344136	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#87656
344099	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89064
344092	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89116
344056	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89123
344043	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#89036
344049	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
343994	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#87438
343982	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#85328
343063	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#88962
342911	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#88962

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
25	26	26

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Error Blob

{
  "ErrorMessage": "System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": true
}

Reproduction Steps

While adding support for Half to System.Text.Json (#88962), I noticed one test triggers the assertion failure consistently on OSX.

The test attempts to deserialize floting-point literals (NaN, infinity, -Infinity) in multiple casings/variants that are invalid for S.T.Json. Error happens on line 883.

runtime/src/libraries/System.Text.Json/tests/Common/NumberHandlingTests.cs

Lines 848 to 883 in 43fe8a8

	[Theory]
	[InlineData("naN")]
	[InlineData("Nan")]
	[InlineData("NAN")]
	[InlineData("+Infinity")]
	[InlineData("+infinity")]
	[InlineData("infinity")]
	[InlineData("infinitY")]
	[InlineData("INFINITY")]
	[InlineData("+INFINITY")]
	[InlineData("-infinity")]
	[InlineData("-infinitY")]
	[InlineData("-INFINITY")]
	[InlineData(" NaN")]
	[InlineData("NaN ")]
	[InlineData(" Infinity")]
	[InlineData(" -Infinity")]
	[InlineData("Infinity ")]
	[InlineData("-Infinity ")]
	[InlineData("a-Infinity")]
	[InlineData("NaNa")]
	[InlineData("Infinitya")]
	[InlineData("-Infinitya")]
	#pragma warning disable xUnit1025 // Theory method 'FloatingPointConstants_Fail' on test class 'NumberHandlingTests' has InlineData duplicate(s)
	[InlineData("\u006EaN")] // "naN"
	[InlineData("\u0020Inf\u0069ni\u0074y")] // " Infinity"
	[InlineData("\u002BInf\u0069nity")] // "+Infinity"
	#pragma warning restore xUnit1025
	public async Task FloatingPointConstants_Fail(string testString)
	{
	string testStringAsJson = $@"""{testString}""";
	string testJson;
	#if NETCOREAPP
	testJson = @$"{{""HalfNumber"":{testStringAsJson}}}";
	await Assert.ThrowsAsync<JsonException>(async () => await Serializer.DeserializeWrapper<StructWithNumbers>(testJson, s_optionsAllowFloatConstants));
	await Assert.ThrowsAsync<JsonException>(async () => await Serializer.DeserializeWrapper<StructWithNumbers>(testJson, s_optionReadFromStr));

Helix log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-88962-merge-9757e201f6374e20b5/System.Text.Json.Tests/1/console.78feb75a.log?helixlogtype=result

Call stack:

/private/tmp/helix/working/9EC208C7/w/A9F108F7/e /private/tmp/helix/working/9EC208C7/w/A9F108F7/e
  Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.Json.Tests (found 7160 of 7231 test cases)
  Starting:    System.Text.Json.Tests (parallel test collections = on, max threads = 6)
Process terminated. Assertion failed.
   at System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar(Byte& charA, Int32 lengthA, Byte& charB, Int32 lengthB) in /_/src/libraries/System.Private.CoreLib/src/System/Globalization/Ordinal.Utf8.cs:line 308
   at System.Number.TryParseFloat[TChar,TFloat](ReadOnlySpan`1 value, NumberStyles styles, NumberFormatInfo info, TFloat& result) in /_/src/libraries/System.Private.CoreLib/src/System/Number.Parsing.cs:line 1229
   at System.Half.TryParse(ReadOnlySpan`1 utf8Text, NumberStyles style, IFormatProvider provider, Half& result) in /_/src/libraries/System.Private.CoreLib/src/System/Half.cs:line 2238
   at System.Text.Json.Serialization.Converters.HalfConverter.TryParse(ReadOnlySpan`1 buffer, Half& result) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 197
   at System.Text.Json.Serialization.Converters.HalfConverter.ReadCore(Utf8JsonReader& reader) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 50
   at System.Text.Json.Serialization.Converters.HalfConverter.ReadNumberWithCustomHandling(Utf8JsonReader& reader, JsonNumberHandling handling, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 115
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs:line 193
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonPropertyInfoOfT.cs:line 308
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Object/ObjectDefaultConverter.cs:line 49
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs:line 258
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.ReadCore.cs:line 51
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.ReadHelper.cs:line 22
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.Span.cs:line 160
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs:line 443
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs:line 55
   at System.Text.Json.Serialization.Tests.JsonSerializerWrapper.StringSerializerWrapper.DeserializeWrapper[T](String json, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/JsonSerializerWrapper.Reflection.cs:line 129
   at System.Text.Json.Serialization.Tests.NumberHandlingTests.<>c__DisplayClass39_0.<<FloatingPointConstants_Fail>b__1>d.MoveNext() in /_/src/libraries/System.Text.Json/tests/Common/NumberHandlingTests.cs:line 883
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) in /_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs:line 38
   at System.Text.Json.Serialization.Tests.NumberHandlingTests.<>c__DisplayClass39_0.<FloatingPointConstants_Fail>b__1()

Author:	Jozkee
Assignees:	-
Labels:	area-System.Text.Json, blocking-clean-ci, Known Build Error
Milestone:	-

[jozkee]

cc @tannergooding

[elinor-fung]

Recent PRs are consistently hitting this on Linux and Windows as well: linux, windows

[EgorBo]

Sounds like it's from #86875 (at least it touched Ordinal.Utf8.cs and various parsing logic including Half)

[markples]

adding some keywords to help github issue search (?)

EqualsIgnoreCaseUtf8_Scalar
Json.Tests
System.Text.Json.SourceGeneration.Roslyn4.4.Tests
Roslyn4.4.Tests
Roslyn4

[EgorBo]

Looks like #88962 is the PR that broke CI by uncovering an existing bug, it seems that the assert was noticed and ignored on macOS 8c3e143