feat: move sampling options (temperature/topP/maxTokens) into modelOptions (#660)

* refactor(ai-openai): read sampling options from modelOptions

* refactor(openai-base): read sampling options from modelOptions in chat-completions base

* refactor(ai-anthropic): read sampling options from modelOptions, drop cast

* fix(ai-anthropic): exempt max_tokens from dropped-key warning

* refactor(ai-gemini): read sampling options from modelOptions

* fix(ai-ollama): read sampling from nested modelOptions.options, drop cast and flat root reads

* refactor(ai-openrouter): read sampling options from modelOptions, drop cast

* refactor(ai): remove root sampling options; modelOptions is the sole sampling surface

* fix(ai): preserve summarize maxLength per-provider + fix otel sampling attribute spellings

* refactor(ai-openrouter): read sampling from modelOptions in responses adapter

* refactor(ai-gemini): read sampling from modelOptions in text-interactions adapter

* test: migrate remaining root sampling usages to modelOptions

* feat(codemods): add move-sampling-to-model-options codemod

* docs: document sampling options under modelOptions + migration guide

* docs(skills): sampling options now live in modelOptions

* chore: changeset for sampling-options-to-modelOptions move

* docs: correct sampling migration framing to breaking change

* ci: apply automated fixes

* fix(sampling): address PR #660 review feedback

Blocking fixes (codemod CI + Ollama silent drops):
- codemod: Prettier-normalize the transform test harness so recast's print
  style no longer diverges from the Prettier-formatted fixtures (20/20 green),
  and add a `test:lib` script to the codemods package so `nx affected` (and
  thus `test:pr`) actually runs the suite instead of skipping it.
- summarize: place the default `temperature` where the wrapped provider reads
  it — nested under `options` for Ollama (a flat value was dropped at the wire
  while still surfacing in OTel). Honor caller-set flat token limits in the
  Ollama branch, and warn instead of silently dropping `maxLength` for an
  unrecognised adapter name.
- code-mode eval harness: nest Ollama `num_predict`/`num_ctx` under `options`.
- examples: route the generic `maxTokens` through provider-native
  `modelOptions` (shared `maxTokensModelOptions` helper for dynamically
  resolved adapters; native keys inline where the adapter is static).

Other review items:
- anthropic: `max_tokens ?? 1024` so an explicit `0` reaches validation
  instead of being coerced to the default.
- openrouter (chat): forward root `metadata` like the responses adapter (+ test).
- otel: `firstNumber` now requires `Number.isFinite` (rejects NaN/Infinity).
- logger: add `InternalLogger.warn`, gated by the `errors` category so
  `debug: false` still silences it.
- docs: fix the migration example to put sampling under `modelOptions`; drop
  leaked trailing tags from the sampling guide; note the removed
  `OllamaTextProviderOptions` export in the changeset; split the codemod
  README's OpenAI row into Responses vs Chat Completions.
- codemod: emit ES6 shorthand when a moved value matches its key; drop an
  unsafe `as RootSamplingKey` cast.

* refactor(ai): share max-token spelling table between summarize and otel

Extract the provider-native max-output-token key spellings into
`utilities/sampling-keys.ts` so the summarize wrapper's caller-limit
detection and the OTel middleware's sampling-attribute extraction read
from one source and cannot drift. Also document the intentional
non-native-spelling caveat in `applyMaxLength`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Tom Beckenham <34339192+tombeckenham@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: 4 people

.changeset/sampling-options-to-model-options.md

@@ -0,0 +1,33 @@
+---
+'@tanstack/ai': minor
+'@tanstack/openai-base': minor
+'@tanstack/ai-openai': minor
+'@tanstack/ai-anthropic': minor
+'@tanstack/ai-gemini': minor
+'@tanstack/ai-grok': minor
+'@tanstack/ai-groq': minor
+'@tanstack/ai-ollama': minor
+'@tanstack/ai-openrouter': minor
+---
+
+**BREAKING:** Sampling options (`temperature`, `topP`, `maxTokens`) have moved off the root of `chat()` / `ai()` / `generate()` and into provider-native `modelOptions`. There is no longer a generic root-level sampling surface — each provider accepts its own native keys, fully typed per model:
+
+- OpenAI (Responses): `modelOptions: { temperature, top_p, max_output_tokens }`
+- Anthropic: `modelOptions: { temperature, top_p, max_tokens }`
+- Gemini: `modelOptions: { temperature, topP, maxOutputTokens }`
+- Grok: `modelOptions: { temperature, top_p, max_tokens }`
+- Groq: `modelOptions: { temperature, top_p, max_completion_tokens }`
+- Ollama: `modelOptions: { options: { temperature, top_p, num_predict } }` (nested)
+- OpenRouter (chat): `modelOptions: { temperature, topP, maxCompletionTokens }`
+
+Middleware no longer sees `temperature`/`topP`/`maxTokens` as first-class fields on `ChatMiddlewareConfig`; mutate `config.modelOptions` (with the provider-native keys above) instead. `metadata` is unaffected and stays at the root.
+
+The public `OllamaTextProviderOptions` type export has also been removed from `@tanstack/ai-ollama`. `modelOptions` is now typed per model — use the exported `OllamaChatModelOptionsByName` map (indexed by model name) or the underlying `ChatRequest` from the `ollama` SDK for arbitrary model strings.
+
+Migrate automatically with the codemod, which resolves the provider from the adapter and rewrites the keys for you:
+
+```bash
+pnpm codemod:move-sampling-to-model-options "src/**/*.{ts,tsx}"
+```
+
+See the [Sampling Options migration guide](https://tanstack.com/ai/latest/docs/migration/sampling-options-to-model-options) for details.

codemods/README.md

@@ -6,9 +6,10 @@ Each codemod lives in its own subdirectory and is named after the migration it c
 
 ## Available codemods
 
-| Codemod                                  | Migrates                                                                                                                                                                                                                                                      |
-| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| [`ag-ui-compliance`](./ag-ui-compliance) | Client-side renames introduced by the AG-UI client/server compliance release: `body` → `forwardedProps` on `useChat` / `ChatClient` / `updateOptions`, Svelte's `updateBody` → `updateForwardedProps`, and `chat({ conversationId })` → `chat({ threadId })`. |
+| Codemod                                                              | Migrates                                                                                                                                                                                                                                                      |
+| -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| [`ag-ui-compliance`](./ag-ui-compliance)                             | Client-side renames introduced by the AG-UI client/server compliance release: `body` → `forwardedProps` on `useChat` / `ChatClient` / `updateOptions`, Svelte's `updateBody` → `updateForwardedProps`, and `chat({ conversationId })` → `chat({ threadId })`. |
+| [`move-sampling-to-model-options`](./move-sampling-to-model-options) | Moves root `temperature` / `topP` / `maxTokens` off `chat()` / `ai()` / `generate()` / `createChatOptions()` into provider-native `modelOptions`, renamed per provider (with ollama nesting under `options`).                                                 |
 
 ## Running a codemod
 

codemods/move-sampling-to-model-options/README.md

@@ -0,0 +1,91 @@
+# `move-sampling-to-model-options`
+
+Moves the root-level convenience sampling props — `temperature`, `topP`, and
+`maxTokens` — off `chat()` / `ai()` / `generate()` / `createChatOptions()`
+calls (imported from `@tanstack/ai`) and into the provider-native
+`modelOptions` object, renaming each one to its provider's canonical option
+name.
+
+This is a **breaking change**: the root-level props have been removed, so run
+this codemod to migrate existing call sites onto the new `modelOptions` shape.
+
+## What it changes
+
+The provider is resolved from the `adapter:` property's factory call (e.g.
+`openaiText('gpt-4o')` → `openai`). Each present root prop is moved into
+`modelOptions` under its provider-specific name:
+
+| Root prop     | openai (Responses)  | anthropic     | gemini            | grok          | groq                    | openrouter            | ollama (nested)       |
+| ------------- | ------------------- | ------------- | ----------------- | ------------- | ----------------------- | --------------------- | --------------------- |
+| `temperature` | `temperature`       | `temperature` | `temperature`     | `temperature` | `temperature`           | `temperature`         | `options.temperature` |
+| `topP`        | `top_p`             | `top_p`       | `topP`            | `top_p`       | `top_p`                 | `topP`                | `options.top_p`       |
+| `maxTokens`   | `max_output_tokens` | `max_tokens`  | `maxOutputTokens` | `max_tokens`  | `max_completion_tokens` | `maxCompletionTokens` | `options.num_predict` |
+
+The `openai` column above is the **Responses** adapter (`openaiText`), whose
+`maxTokens` key is `max_output_tokens`. The **Chat Completions** adapter
+(`openaiChatCompletions`) instead uses `max_tokens`, and is _not_ auto-resolved
+by this codemod — those call sites are left untouched and reported, so migrate
+them by hand: `temperature → temperature`, `topP → top_p`,
+`maxTokens → max_tokens`.
+
+For **ollama**, the renamed keys are nested inside a `options` object **within**
+`modelOptions` (e.g. `modelOptions: { options: { temperature, num_predict } }`).
+
+### Example (openai)
+
+```ts
+// before
+chat({
+  adapter: openaiText('gpt-4o'),
+  messages,
+  temperature: 0.3,
+  maxTokens: 100,
+})
+
+// after
+chat({
+  adapter: openaiText('gpt-4o'),
+  messages,
+  modelOptions: {
+    temperature: 0.3,
+    max_output_tokens: 100,
+  },
+})
+```
+
+If `modelOptions` already exists (as an object literal), the renamed keys are
+merged into it. Original value expressions are preserved; a shorthand prop
+(`{ temperature }`) whose provider key is unchanged stays shorthand
+(`{ temperature }`), and one whose key is renamed becomes `newKey: temperature`.
+
+## Running it
+
+From this repo:
+
+```bash
+pnpm codemod:move-sampling-to-model-options "src/**/*.{ts,tsx}"
+```
+
+Or directly against the published transform — no clone needed:
+
+```bash
+npx jscodeshift \
+  --parser=tsx \
+  -t https://raw.githubusercontent.com/TanStack/ai/main/codemods/move-sampling-to-model-options/transform.ts \
+  src/**/*.{ts,tsx}
+```
+
+Add `--dry --print` to preview the rewrite without modifying files.
+
+## Report / skip behavior
+
+The codemod never partially transforms a single call. It leaves the call
+untouched and emits an `api.report(...)` message in these cases:
+
+- **Unresolvable adapter** — no `adapter` prop, the adapter value isn't a
+  recognized provider-factory call (e.g. `makeAdapter()`), or it's
+  dynamic/spread.
+- **`modelOptions` is not a plain object literal** — e.g. a spread or an
+  identifier reference.
+- **Key conflict** — a target renamed key already exists in `modelOptions`
+  (or in `modelOptions.options` for ollama). Resolve these by hand.

codemods/move-sampling-to-model-options/__testfixtures__/anthropic-merge.input.ts

@@ -0,0 +1,12 @@
+import { chat } from '@tanstack/ai'
+import { anthropicText } from '@tanstack/ai-anthropic'
+
+export function run(messages: Array<unknown>) {
+  const temperature = 0.5
+  return chat({
+    adapter: anthropicText('claude-3-5-sonnet-latest'),
+    messages,
+    modelOptions: { top_k: 40 },
+    temperature,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/anthropic-merge.output.ts

@@ -0,0 +1,15 @@
+import { chat } from '@tanstack/ai'
+import { anthropicText } from '@tanstack/ai-anthropic'
+
+export function run(messages: Array<unknown>) {
+  const temperature = 0.5
+  return chat({
+    adapter: anthropicText('claude-3-5-sonnet-latest'),
+    messages,
+
+    modelOptions: {
+      top_k: 40,
+      temperature,
+    },
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/conflict.input.ts

@@ -0,0 +1,14 @@
+// Conflict case: root `temperature` AND `modelOptions.temperature` are
+// both present. The codemod must leave the whole call alone and report.
+
+import { chat } from '@tanstack/ai'
+import { openaiText } from '@tanstack/ai-openai'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: openaiText('gpt-4o'),
+    messages,
+    modelOptions: { temperature: 0.9 },
+    temperature: 0.3,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/conflict.output.ts

@@ -0,0 +1,14 @@
+// Conflict case: root `temperature` AND `modelOptions.temperature` are
+// both present. The codemod must leave the whole call alone and report.
+
+import { chat } from '@tanstack/ai'
+import { openaiText } from '@tanstack/ai-openai'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: openaiText('gpt-4o'),
+    messages,
+    modelOptions: { temperature: 0.9 },
+    temperature: 0.3,
+  })
+}

codemods/move-sampling-to-model-options/__testfixtures__/create-chat-options.input.ts

@@ -0,0 +1,8 @@
+import { createChatOptions } from '@tanstack/ai'
+import { openaiText } from '@tanstack/ai-openai'
+
+export const options = createChatOptions({
+  adapter: openaiText('gpt-4o'),
+  temperature: 0.2,
+  topP: 0.8,
+})

codemods/move-sampling-to-model-options/__testfixtures__/create-chat-options.output.ts

@@ -0,0 +1,11 @@
+import { createChatOptions } from '@tanstack/ai'
+import { openaiText } from '@tanstack/ai-openai'
+
+export const options = createChatOptions({
+  adapter: openaiText('gpt-4o'),
+
+  modelOptions: {
+    temperature: 0.2,
+    top_p: 0.8,
+  },
+})

codemods/move-sampling-to-model-options/__testfixtures__/gemini-rename.input.ts

@@ -0,0 +1,11 @@
+import { chat } from '@tanstack/ai'
+import { geminiText } from '@tanstack/ai-gemini'
+
+export function run(messages: Array<unknown>) {
+  return chat({
+    adapter: geminiText('gemini-1.5-pro'),
+    messages,
+    topP: 0.9,
+    maxTokens: 512,
+  })
+}