Skip to content

Encoding span polyfills throw on empty spans when unsafe is enabled #569

Description

@thomhurst

Description

For target frameworks without the native Encoding span overloads, the unsafe polyfills pin spans and forward them to the pointer overloads:

fixed (char* charsPtr = chars)
fixed (byte* bytesPtr = bytes)
{
    return target.GetBytes(charsPtr, chars.Length, bytesPtr, bytes.Length);
}

Pinning an empty span produces a null pointer. The pointer overloads reject null pointers even when the corresponding length is zero, so these polyfills do not match the native span overload semantics.

This was observed with Polyfill 10.11.2 in a netstandard2.0 library with AllowUnsafeBlocks=true, running on a .NET 8 host. The same implementation is still present on main / 11.0.0:

Reproduction

Compile these calls against a target where Polyfill supplies the span overloads, with unsafe blocks enabled:

using System;
using System.Text;

ReadOnlySpan<char> emptyChars = default;
ReadOnlySpan<byte> emptyBytes = default;
Span<byte> destination = new byte[1];

Encoding.UTF8.GetByteCount(emptyChars);
Encoding.UTF8.GetBytes(emptyChars, destination);
Encoding.UTF8.GetString(emptyBytes);

Expected behavior

Matching the native span overloads:

  • GetByteCount(emptyChars) returns 0
  • GetBytes(emptyChars, destination) returns 0
  • GetString(emptyBytes) returns string.Empty

Actual behavior

ArgumentNullException is thrown from the underlying pointer overload, for example:

System.ArgumentNullException: Array cannot be null. (Parameter 'chars')
  at System.Text.UTF8Encoding.GetBytes(Char* chars, ...)
  at Polyfills.Polyfill.GetBytes(
      Encoding target,
      ReadOnlySpan<char> chars,
      Span<byte> bytes)

An empty destination span can produce the same semantic mismatch: a null bytes pointer is passed instead of a non-null pointer with length zero.

Impact

This caused a real runtime failure in Dekaf. Its netstandard2.0 Kafka consumer writes an empty compact string (the default client rack ID) in every fetch request. Every prefetch threw, so producing worked but consuming always timed out:

Dekaf currently carries an EncodingCompat wrapper that short-circuits empty spans.

Related methods

The same unsafe pin-and-forward pattern appears in:

  • Encoding.GetChars(ReadOnlySpan<byte>, Span<char>)
  • Encoding.GetCharCount(ReadOnlySpan<byte>)
  • Encoder.GetBytes(ReadOnlySpan<char>, Span<byte>, bool)

These likely need the same empty-span handling and regression coverage.

Suggested fix

Short-circuit empty input where the result is unconditionally empty. For zero-length destination spans with non-empty input, use a non-null dummy pointer with length zero (or equivalent) so the underlying overload preserves its normal capacity and fallback behavior.

Please add empty-source and empty-destination coverage to the unsafe test project for all affected span polyfills.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions