Add vector(float16) support

[apoorvdeshmukh]

Adds support for the float16 base type of the SQL Server vector data type, on both .NET and .NET Framework.

Behaviour

float16 is exchanged in its binary form only when a connection opts in, through the new Vector Type Support keyword (off | v1 | v2, default v1). This mirrors vectorTypeSupport in the JDBC driver, including its default. At v1 a float16 column is returned as a varchar(max) containing a JSON array, exactly as today, so upgrading the driver changes nothing until an application asks for v2.

At v2:

	.NET	.NET Framework
GetValue / GetFieldType	SqlVector<Half>	string (JSON array)
Strongly typed read	GetSqlVector<Half>	GetSqlVector<float> — widening, which is exact
Write	SqlVector<Half>, SqlVector<float>, or a JSON string	SqlVector<float> or a JSON string

System.Half does not exist on .NET Framework, so a float16 column is surfaced there as its JSON rendering rather than by substituting a different base type. A caller that wants a strongly typed value asks for SqlVector<float> explicitly.

Column metadata

A vector column now reports its base type and dimension count through the existing DbColumn indexer, with no new API:

DbColumn column = reader.GetColumnSchema()[0];

string baseType   = (string)column["VectorBaseType"];    // "float16" | "float32"
int    dimensions = (int)   column["VectorDimensions"];

This is the only way to distinguish the two base types when a float16 column is surfaced as a JSON string, because GetFieldType reports string for such a column just as it does for a varchar one.

Bulk copy

The INSERT BULK statement states the destination column's base type, and the server then requires a binary payload of exactly that width and performs no conversion within the data stream. Measured at every negotiated version: a text declaration for a vector column is refused with Invalid column type from bcp client, and text sent under a vector declaration is refused with a length mismatch. Only a client which never negotiates the feature extension may send text.

So a textual source is parsed into the destination's base type by the driver, and a payload read from another vector column keeps its own base type — copying between columns of different base types is reported by the server rather than silently narrowed. Both match the JDBC driver.

Public API

One addition: the SqlVectorTypeSupport enum and SqlConnectionStringBuilder.VectorTypeSupport. SqlVector<T> is unchanged.

Tests

• float16 added to the existing generic native vector suite, run over a v2 connection
• the same suite run through SqlVector<float> on every framework, which is the .NET Framework representation and where the hand written binary16 codec is the production path
• version negotiation per keyword value, including the client's own version ceiling
• the binary16 codec validated against System.Half across all 65,536 binary16 and all 4,294,967,296 binary32 patterns

Checklist

• Tests added or updated
• Public API changes documented
• Ensure no breaking changes introduced — the default keeps the current representation

Notes for reviewers: the second commit reworks bulk copy and negotiation following a design review, so it is worth reading over the first. One review thread is left open pending that discussion.

[Copilot]

Pull request overview

Adds end-to-end support for SQL Server vector(N, float16) by negotiating VECTORSUPPORT feature extension version 2, introducing an IEEE-754 binary16 codec, and wiring float16 handling through SqlVector<T> read/write paths (including bulk copy), with accompanying docs and tests.

Changes:

• Negotiate vector feature extension v2 and track negotiated vector capability version (float32/float16) on the connection.
• Add float16 vector support across SqlVector<T>, SqlDataReader, SqlBuffer, SqlParameter, SqlCommand, and SqlBulkCopy, including payload conversion for bulk copy.
• Add unit/manual tests plus docs/snippets/sample updates; expose vector base type + dimensions via DbColumn indexer and register vector in the DataTypes schema collection.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/Microsoft.Data.SqlClient/tests/UnitTests/SimulatedServerTests/ConnectionTests.cs	Updates simulated negotiation tests for vector feature extension v2.
src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlTypes/SqlVectorTest.cs	Adds float16 construction/rendering tests and payload conversion tests.
src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlTypes/Float16ConverterTest.cs	New unit tests validating binary16 codec (incl. exhaustive bit-pattern validation on .NET).
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/VectorTest/VectorFloat16BehaviourTests.cs	Manual tests for float16-specific behaviors (read/write/cross-base-type/bulk copy).
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/VectorTest/VectorColumnMetadataTests.cs	Manual tests for vector column metadata + schema collection registration.
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/VectorTest/NativeVectorFloat16Tests.cs	Manual typed tests for SqlVector<Half> on .NET.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlTypes/SqlVector.cs	Implements float16 support in SqlVector<T>, adds JSON ToString(), payload widening/conversion helpers.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/TdsEnums.cs	Adds vector version constants and sets max supported vector version to float16 (v2).
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlParameter.cs	Handles float16 vector return/coercion paths (Half on .NET, widening on netfx).
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlMetaDataFactory.DataTypes.cs	Registers vector in GetSchema("DataTypes").
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlEnums.cs	Adds float16 vector element type and element-size mapping; meta type inference includes SqlVector<Half> on .NET.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlDbColumn.cs	Exposes VectorBaseType/VectorDimensions via DbColumn indexer.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlDataReader.cs	Adds float16 field-type mapping and broadens GetSqlVector<T> to allow Half on .NET.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.cs	Emits (N, float16) parameter declaration when needed (keeps float32 declaration unchanged).
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs	Emits float16 vector type in INSERT BULK declaration and converts payload base type to match destination.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBuffer.cs	Centralizes float16 vector rendering/value-shaping across frameworks.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/Connection/SqlConnectionInternal.cs	Records negotiated vector feature version on FEATUREEXTACK.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/Connection/ConnectionCapabilities.cs	Replaces bool flag with VectorVersion and derived float32/float16 capability properties.
src/Microsoft.Data.SqlClient/src/Microsoft/Data/Common/Float16Converter.cs	New internal binary16<->binary32 conversion implementation.
src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlTypes.cs	Updates reference surface to include SqlVector<T>.ToString() override.
doc/snippets/Microsoft.Data.SqlTypes/SqlVector.xml	Documents float16 support, size constraints, and ToString() JSON rendering.
doc/samples/SqlVectorFloat16Example.cs	New sample demonstrating float16 vectors (insert/read/metadata).
.github/instructions/features.instructions.md	Updates repo feature reference docs for float16 vector base type and negotiation versions.

Suppressed comments (1)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlTypes/SqlVector.cs:420

• ConvertPayloadElementType doesn’t validate the vector header magic/version bytes before using the length and element type fields. This can cause non-vector payloads to be converted (or to fail later with less appropriate exceptions). Validate VecHeaderMagicNo/VecVersionNo up front, consistent with GetCountsOrThrow.

        if (tdsBytes.Length < TdsEnums.VECTOR_HEADER_SIZE)
        {
            throw ADP.InvalidVectorHeader();
        }

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated no new comments.

Suppressed comments (4)

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs:1755

• SqlTypes.SqlVector<float> doesn’t resolve to any namespace/type in this file (there’s no using SqlTypes = ... and no SqlTypes namespace). This should be fully qualified to Microsoft.Data.SqlTypes.SqlVector<float> (or add an alias) to avoid a compile error.

            // The payload is converted directly rather than through a strongly typed vector,
            // so that .NET Framework, which has no System.Half, can also write to float16
            // destinations.
            return SqlTypes.SqlVector<float>.ConvertPayloadElementType(payload, destinationElementType);

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlTypes/SqlVector.cs:154

• FromTdsPayload reads header fields (element type/length) without validating the vector magic/version bytes. This makes the widening path accept malformed payloads that GetCountsOrThrow would reject.

        if (tdsBytes.Length < TdsEnums.VECTOR_HEADER_SIZE)
        {
            throw ADP.InvalidVectorHeader();
        }

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlTypes/SqlVector.cs:420

• ConvertPayloadElementType should validate the vector header magic/version before interpreting element type and length; otherwise malformed byte[] values can be converted and sent on the wire rather than failing fast with InvalidVectorHeader.

        if (tdsBytes.Length < TdsEnums.VECTOR_HEADER_SIZE)
        {
            throw ADP.InvalidVectorHeader();
        }

doc/samples/SqlVectorFloat16Example.cs:146

• These interpolated strings won’t compile because the expression uses double quotes (e.g., column["VectorBaseType"]) inside a double-quoted string literal. Escape the quotes (or assign to a local variable) before interpolating.

        Console.WriteLine($"\nColumn base type:  {column["VectorBaseType"]}");
        Console.WriteLine($"Column dimensions: {column["VectorDimensions"]}");

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated no new comments.

Suppressed comments (2)

doc/samples/SqlVectorFloat16Example.cs:146

• These interpolated strings won’t compile because the expression contains a string literal with double quotes (e.g., column["VectorBaseType"]) which terminates the outer interpolated string. Assign the indexer results to variables (or constants) first, then interpolate those variables.

        Console.WriteLine($"\nColumn base type:  {column["VectorBaseType"]}");
        Console.WriteLine($"Column dimensions: {column["VectorDimensions"]}");

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlParameter.cs:2408

• This SqlVector special-case is redundant/unreachable because SqlVector implements ISqlVector (so it will already be handled by the earlier value is ISqlVector branch). Keeping the extra branch increases maintenance burden and risks diverging behavior.

                    else if (currentType == typeof(SqlVector<Half>))
                    {
                        value = ((ISqlVector)value).VectorPayload;
                    }
                    #endif

[saurabh500]

Summary

This adds vector(N, float16) by advertising VECTORSUPPORT v2, introducing a hand-written binary16 codec, teaching SqlVector<T> about System.Half, and switching vector→vector bulk copy to a raw-payload transfer. The engineering is careful, and I want to call out specifically that the endianness and element-size arithmetic is correct throughout — I went looking for a missed (ColumnSize - 8) / 4 and there isn't one. The commit split is clean and the rationale in the description is unusually good.

I have one blocking correctness issue, plus a set of suggestions. Inline comments carry the detail and suggested diffs; this is the map.

Blocking

• SqlBuffer.GetSqlVector<T>() succeeds or throws depending on the row's nullness. The IsNull branch builds a vector for any T without consulting the column's base type, while the non-null branch validates. GetSqlVector<Half>() over a float32 column therefore returns Null for NULL rows and throws NotSupportedException for non-NULL rows in the same result set. Data-dependent rather than schema-dependent, so it is hard to find in testing and impossible to guard against in caller code.

Suggestions

• The codec is not bit-exact against System.Half for NaN, contrary to the description, and the tests are written to step around exactly that case (continue / IsNaN-only). float.NaN carries the sign bit, so widening flips the sign of every NaN relative to System.Half; narrowing canonicalises payloads that (Half)float preserves. Either match System.Half or pin the canonical form with explicit assertions and drop the "bit-exact" claim.
• Bulk copy's vector case uses metadata.scale rather than the scale local that the surrounding code establishes for encrypted columns.
• The widening path skips the magic-number and version validation that the matching path gets via GetCountsOrThrow.
• Capabilities.Float16VectorType is never read anywhere in src/, so a SqlVector<Half> on a v1-negotiated connection fails server-side rather than client-side. (Float32VectorType was already dead in main; this adds a second.)
• The negotiation theory's 0x3 case doesn't test what its comment claims — the simulated server caps the ack itself, so the client's own ceiling check at SqlConnectionInternal.cs:1660 stays untested.
• Docs say narrowing "fails for values outside its range"; the code saturates to ±Infinity and leaves it to the server.
• MetaData is dereferenced without a null check in CreateSourceColumnMetadata.

Things I checked and found correct

Worth recording, since they're the parts most likely to be wrong in a change like this:

• Subnormal widening, signed zero, overflow to infinity, the flush-to-zero boundary (2⁻²⁵ ties to even → 0, 2⁻²⁶ flushes), binary32 subnormal inputs, and the rounding carry into the exponent are all correct. Compiling Manual* on every TFM so the netfx path is under test on .NET too is a good arrangement.
• Bulk copy null handling is correct as-is — GetValueFromSourceRow returns DBNull.Value with isNull = true and ConvertValue returns before ConvertVectorToBaseType. Note a5de733c3 is test-only; it documents behaviour that already worked rather than fixing anything.
• The raw-payload path can't be taken by DataTable, DataRow[], or non-SqlClient DbDataReader sources (they keep ValueMethod.GetValue), and reordering column mappings are safe because sourceOrdinal is the mapped ordinal.
• No unacknowledged breaking changes beyond the three listed: GetDataTypeName is unchanged, GetFieldType/GetProviderSpecificFieldType both route through the single new GetVectorFieldType so they stay consistent, SqlMetaDataFactory.DataTypes only adds a row (gated on MinimumVersionKey), the float32 declaration is deliberately unchanged, and SqlDbColumn's new indexer falls through to base[property].
• No shared mutable state: Float16Converter is stateless and SqlVector<T> is a readonly struct with no static caches.

On testing

Taking as given that CI has no float16-capable server and no Azure SQL DB connectivity, so the manual suite is the only gate that will ever run — I looked at whether it is complete enough for a lab run rather than whether CI covers it. It is substantial: 11 behaviour tests plus the inherited NativeVectorTestsBase matrix, and the sample data is well chosen (Half.MaxValue, Half.Epsilon, -0.0f, exactly-representable eighths). Gaps I'd close:

1. .NET Framework gets none of the NativeVectorTestsBase matrix — NativeVectorFloat16Tests.cs is entirely #if NET. That is where the hand-rolled Manual* codec is the production path.
2. No async coverage for the new representation on any framework, and none at all for float16 on netfx.
3. DataTestUtility.CheckVectorFloat16Supported fails open through the code under test. It reads the probe vector with GetString + JsonSerializer.Deserialize and catches JsonException → false. A driver regression that produces malformed JSON silently skips the whole float16 suite green. Since the manual run is the only gate, that is the wrong failure mode. (Outside this diff, so no inline comment — but worth fixing alongside. It also leaves PREVIEW_FEATURES = ON on the shared test database as a side effect.)
4. Two range tests assert only that some SqlException was thrown; they'd pass on an unrelated failure, and they can't distinguish "client rejects" from "client saturates and server rejects" — which is exactly the ambiguity in the doc wording above.
5. Nothing covers the blocking issue: reading a float32 column as SqlVector<Half>, for a NULL and a non-NULL row.
6. Bulk copy with a dimension-count mismatch between source and destination is uncovered. Mitigating: float32→float32 now routes through the new raw-payload path too and is covered by the existing NativeVectorFloat32Tests, which runs against any vector-capable server — so regression risk to shipped functionality is covered.

Minor

• SqlVector<T>.ToString() changing for existing SqlVector<float> callers is justified and correctly surfaced in the ref assembly and docs, but it is unrelated to float16 — it wants its own release-note entry as a behavioural break, not just an API-list line. Related: GetString() uses JsonSerializer.Serialize, which throws on NaN/Infinity by default, and on .NET Framework that is now the default GetValue() path.
• ConvertPayloadElementType is internal static on SqlVector<T> but never uses T, so callers write SqlVector<float>.ConvertPayloadElementType(...), which reads as though it returns a float32 result.
• On .NET Framework the reader surfaces a float16 column as string while an output parameter surfaces it as SqlVector<float>. Self-consistent, but worth documenting.
• Preprocessor directives in the new code are indented to the surrounding block; the dominant style in these files is column 0.
• ConnectionCapabilities.cs:179 says vectors were "introduced in SQL Server 2022" — pre-existing, but the new Float16VectorType doc sits right beside it.
• Worth confirming the doc/samples build resolves the locally-packed driver: SqlVectorFloat16Example.cs references SqlVector<Half> and GetSqlVector<Half>, which exist in no released package.

_{Review assisted by GitHub Copilot; findings verified against the code at a5de733c3.}

[saurabh500]

@apoorvdeshmukh and @cheenamalhotra, I think we will need to call this API a known limitation for preventing backward migration from .Net Runtime to NetFx.

I believe this is a known reasonable compromise.

[saurabh500]

BTW, I saw changes to ref assembly. Should I expect two copies of changes, one for netcore and another for NetFx? I am curious about how the APIs will show up in contract assemblies targeting 2 different frameworks.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Note

This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.

[Copilot]

Pull request overview

Copilot reviewed 39 out of 40 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs:1232

• The comment says a base-type mismatch between source and destination is resolved by conversion, but the VectorPayload path preserves the raw payload (and therefore its base type). Mismatches are expected to be reported by the server, and only textual sources are rewritten client-side.

This issue also appears on line 1459 of the same file.

                                // Transfer the vector as its raw payload, so that no value is
                                // lost to an intermediate representation and no larger textual
                                // form is sent. Any difference in base type between the source
                                // and the destination is resolved when the value is converted.

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs:1463

• The XML doc comment describes vector bulk-copy values being transferred as text and converted by the server, and implies the destination column is declared as varchar(max). The implementation actually coerces JSON strings to a vector payload client-side and sends binary; the server does not parse JSON within the bulk-copy stream.

        /// <summary>
        /// Whether a vector destination column is supplied from a textual source, in which
        /// case the value is transferred as text and converted by the server.
        /// </summary>
        /// <remarks>

[Copilot]

Pull request overview

Copilot reviewed 40 out of 41 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file

[Copilot]

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/VectorTest/VectorColumnMetadataTests.cs:158

• Inside the row loop, Assert.Contains(baseType, new[] { "float16", "float32" }) is redundant/confusing because baseType is already asserted to be "float16" above and never changes per row. Keeping this check suggests the query returns mixed base types, but it doesn’t.

[Copilot]

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• src/Microsoft.Data.SqlClient/src/Resources/Strings.Designer.cs: Generated file

[codecov]

Codecov Report

❌ Patch coverage is 72.93497% with 154 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.66%. Comparing base (f3db5d4) to head (45f4fd7).
⚠️ Report is 37 commits behind head on main.

Files with missing lines	Patch %	Lines
...nt/src/Microsoft/Data/SqlTypes/SqlVectorPayload.cs	21.05%	45 Missing ⚠️
...mon/ConnectionString/VectorTypeSupportUtilities.cs	62.16%	28 Missing ⚠️
...Client/src/Microsoft/Data/SqlClient/SqlBulkCopy.cs	67.46%	27 Missing ⚠️
...SqlClient/src/Microsoft/Data/SqlTypes/SqlVector.cs	73.61%	19 Missing ⚠️
...ient/src/Microsoft/Data/SqlClient/SqlDataReader.cs	76.92%	9 Missing ⚠️
...Client/src/Microsoft/Data/SqlClient/SqlDbColumn.cs	80.76%	5 Missing ⚠️
...lient/src/Microsoft/Data/SqlClient/SqlParameter.cs	16.66%	5 Missing ⚠️
...qlClient/src/Microsoft/Data/SqlClient/SqlBuffer.cs	81.81%	4 Missing ⚠️
...c/Microsoft/Data/SqlClient/SqlConnectionOptions.cs	78.57%	3 Missing ⚠️
...SqlClient/src/Microsoft/Data/SqlClient/SqlEnums.cs	50.00%	3 Missing ⚠️
... and 3 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4501      +/-   ##
==========================================
- Coverage   64.60%   63.66%   -0.94%     
==========================================
  Files         288      287       -1     
  Lines       44046    68404   +24358     
==========================================
+ Hits        28455    43551   +15096     
- Misses      15591    24853    +9262     

Flag	Coverage Δ
CI-SqlClient	?
PR-SqlClient-Project	63.66% <72.93%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.