Skip to content

llms/ollama: WithThink has no effect — think is serialized inside "options" (API expects top-level) and omitempty drops false #1514

Description

@lunetics

Two independent defects in the Ollama integration make WithThink (and, on current main, the ThinkingMode-derived Think flag) a silent no-op.

1. Wrong placement: think is sent inside options, but the Ollama API defines it as a top-level field.

Think is declared in the runner-Options struct (types.go#L170 @v0.1.14, unchanged on main):

PenalizeNewline  bool    `json:"penalize_newline,omitempty"`
Think            bool    `json:"think,omitempty"` // Ollama 0.9.0+ reasoning mode

ChatRequest serializes that struct under "options" and has no top-level think field. The Ollama API expects think at the top level of both /api/chat and /api/generate (docs/api.md: "think: (for thinking models) should the model think before responding? Can be a boolean or a thinking level"). Unknown keys inside options are ignored by the server.

The repo's own recorded fixture shows the resulting wire format (llms/ollama/testdata/TestWithThink.httprr):

{"model":"gemma3:1b","messages":[...],"format":"","options":{"temperature":0,"think":true}}

2. omitempty on a plain bool: WithThink(false) serializes to nothing at all.

false is the zero value, so even with the placement fixed, "explicitly disable thinking" could never be expressed. For thinking models (qwen3, deepseek-r1, …) the server default is to think — false is precisely the value users need to send.

Net effect: WithThink(true) → ignored key inside options; WithThink(false) → omitted entirely; both silent. This likely also explains #1460 — the request-side flag never reaches the server, so all thinking-related options appear dead.

Measured impact: paperless-gpt exposes OLLAMA_THINK backed by WithThink; the flag has no effect. With qwen3:14b our metadata pipeline ran at ~3 min/document (model thinking) vs ~8 s/document once think: false actually reached the server top-level (injected via a proxy). In a paired n=23 run (strict-JSON classification, qwen3:14b), disabling thinking was equal or better on accuracy (87.0 % vs 82.6 % on the main field).

Suggested fix — move Think out of Options into ChatRequest (and the generate request) as a pointer, so unset ≠ false, and populate it from the existing option paths (WithThink, the ThinkingMode mapping in ollamallm.go):

type ChatRequest struct {
	Model     string     `json:"model"`
	Messages  []*Message `json:"messages"`
	Stream    bool       `json:"stream,omitempty"`
	Format    string     `json:"format"`
	KeepAlive string     `json:"keep_alive,omitempty"`
	Think     *bool      `json:"think,omitempty"` // top-level per Ollama API; pointer keeps unset ≠ false
	Options   Options    `json:"options"`
}

(A follow-up could widen the type to bool | "low" | "medium" | "high" per current Ollama semantics.)

Affected: v0.1.14 and current main.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions