Skip to content

feat(#3996): resolve image output capability from models.dev - #4019

Draft
aheritier wants to merge 2 commits into
gemini-request-diagnosticsfrom
gemini-output-capability
Draft

feat(#3996): resolve image output capability from models.dev#4019
aheritier wants to merge 2 commits into
gemini-request-diagnosticsfrom
gemini-output-capability

Conversation

@aheritier

@aheritier aheritier commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

What

Resolves Gemini image-output capability from models.dev, wires it into model/config validation, and documents/examples the capability.

Why

Allows image-output configuration to be validated from authoritative model metadata instead of hard-coded assumptions.

Validation

task test; config, schema, modelinfo, capability-resolution, and output-capability tests.

Test instructions

Run:

task test

Use examples/gemini_image_output.yaml with a Gemini model that advertises image output, then repeat with a model that does not. Expected: the capable model configuration is accepted and the incapable model configuration is rejected with a clear validation error.

@aheritier aheritier added area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/testing Test infrastructure, CI/CD, test runners, evaluation kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Aug 22, 2026
@aheritier
aheritier force-pushed the gemini-output-capability branch 2 times, most recently from ebd9306 to 33c6981 Compare August 24, 2026 15:12
@aheritier
aheritier force-pushed the gemini-output-capability branch from 33c6981 to 5bb315a Compare August 25, 2026 20:24
@aheritier
aheritier force-pushed the gemini-output-capability branch 2 times, most recently from 35efc65 to 5bb315a Compare September 1, 2026 16:19
@aheritier
aheritier force-pushed the gemini-output-capability branch from 5bb315a to d1bcf0a Compare September 1, 2026 16:29
aheritier

This comment was marked as resolved.

Add ModelConfig.OutputCapabilities (output_capabilities.image in YAML), a latest-only tri-state override for a model's generative output capability. Explicit false has highest precedence, explicit true follows, and an omitted flag — including output_capabilities: {} — is left for models.dev resolution in the following commit. No capability is ever inferred from the model name.

Reject output_capabilities on first_available selectors because one block cannot unambiguously describe every candidate. Update the schema, model docs, and example, and cover true, false, omitted, and empty-block YAML states, shorthand behavior, cloning, schema strictness, and selector validation.
Resolve image-output capability with one explicit precedence chain: false disables it, true enables it, and an omitted image flag — whether output_capabilities is absent or {} — enables it only when the exact known models.dev record lists image in Modalities.Output. Unknown models, unavailable catalogue data, and lookup failures stay off.

Keep lookup case-insensitive and bounded by the existing models.dev timeout. Capability is never inferred from provider or model-name patterns.
@aheritier
aheritier force-pushed the gemini-output-capability branch from a8c6997 to 54e8365 Compare September 7, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config For configuration parsing, YAML, environment variables area/docs Documentation changes area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/testing Test infrastructure, CI/CD, test runners, evaluation kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant