Enable Nullable options for CreateSchemaJson to use OpenAPI recommendation.

[rogerbarreto]

Description

Currently when a schema is generated for nullable properties, the type is defined as an array ["null","<property_type>"].

This does not follow the standard for OpenAPI 3.0 spec:

• https://spec.openapis.org/oas/v3.0.3#schema-object
  type - Value MUST be a string. Multiple types via an array are not supported.
4.7.24.2 Fixed Fields - Should have an additional field name "nullable": boolead

This becomes a problem when API's for Structured Outputs follows the OpenAPI 3.0 standard and some workarounds needs to be put in place as in this PR, having a setting, would be ideal.

• .Net: Fix Nullable Bug Gemini Schema Generation microsoft/semantic-kernel#11584

Reproduction Steps

Reproduction Code

using Microsoft.Extensions.AI;

var schema = AIJsonUtilities.CreateJsonSchema(typeof(Class1));

Console.WriteLine(schema.GetRawText());

class Class1 
{
    public string? Age { get; set; }
}

Output:

{
  "type": "object",
  "properties": {
    "age": {
      "type": [
        "string",
        "null"
      ]
    }
  },
  "additionalProperties": false,
  "required": [
    "age"
  ]
}

Expected behavior

A SchemaGeneration option that comply with OpenAPI 3.0 spec, generating one type and an additional Nullable boolean field in the schema.

Following the RFC (https://spec.openapis.org/oas/v3.0.3#schema-object)

Output:

{
  "type": "object",
  "properties": {
    "age": {
      "type": "string",
      "nullable": true
    }
  },
  "additionalProperties": false,
  "required": [
    "age"
  ]
}

Actual behavior

Output:

{
  "type": "object",
  "properties": {
    "age": {
      "type": [
        "string",
        "null"
      ]
    }
  },
  "additionalProperties": false,
  "required": [
    "age"
  ]
}

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

When using other types (not string) this gets trickier as the array contains multiple types for the same property.

Using int? example:

{
  "type": "object",
  "properties": {
    "age": {
      "type": [
        "string",
        "integer", 
        "null"
      ]
    }
  },
  "additionalProperties": false,
  "required": [
    "age"
  ]
}

[eiriktsarpalis]

Using arrays in the type keyword is valid per the JSON schema specification, and this is generally the spec we try to remain compatible with by default. The nullable keyword was a OpenAPI addition that has been deprecated as of OpenAI 3.1 in favor of the standardized type keyword.

Is there a strong need for supporting older versions of OpenAPI?

[rogerbarreto]

@eiriktsarpalis We do have Google API using the old way, would be desireable to have a schemaOptions to set what would be the version compliance of the schema generated, that would be very handy.

[eiriktsarpalis]

I'd be reluctant to do this, we did something similar to cater to OpenAI requirements and we're now in the process of rolling this back because of the ripple effect it causes to unrelated services. I suspect it should be possible to achieve this using a transformer delegate, would it be possible to give that a go?

[eiriktsarpalis]

Tagging @captainsafia @darrelmiller who are more familiar with OpenAPI than me. Do you believe generating schemas using the nullable keyword instead of type is warranted nowadays?

[rogerbarreto]

I'd be reluctant to do this, we did something similar to cater to OpenAI requirements and we're now in the process of #6266 because of the ripple effect it causes to unrelated services. I suspect it should be possible to achieve this using a transformer delegate, would it be possible to give that a go?

Not sure if this ideal, requiring all the callers to implement a delegate, to understand how it works. Much simplier would be a tag for conformity (version) or a new flag for how handling nullables vs types convention. Were it would default to latest in which would not cause any breaking change in existing code.

[eiriktsarpalis]

Using conformance modes for schema generation is something I've been discussing with @stephentoub but it hasn't gained traction so far. We've instead resorted to users opting in to whatever transformations they need to apply themselves.

[jeffhandley]

Thank you both. We won't try to fix this in a release for this/next week. But we need to resolve this issue in some way before going GA, even if we just have a workaround available without an MEAI change to add a flag or conformance mode.