Skip to content

Constrained tool-call decoding: validate (local vLLM) + wire #868

Description

@anandgupta42

Context

provider/constrained.ts (grammar/JSON-Schema constrained decoding for local providers) was split out of #858 so the validated tool-retrieval lever could land clean. Code + unit tests live on branch feat/constrained-decoding.

Work

  1. Validate: run a local vLLM guided-decoding A/B (constrained ON vs OFF) on a tool-heavy task set; confirm it improves tool-call parse-rate / reduces malformed calls at no quality cost.
  2. Wire: re-add the flag-gated (ALTIMATE_CONSTRAINED_TOOLCALLS, local-providers-only, toolChoice: "required") block into session/llm.ts (marker-wrapped).
  3. Open the PR once it actually does something measurable.

Do NOT open a PR with the flag wired-but-unvalidated or present-but-unwired (the no-op-flag anti-pattern). Branch: feat/constrained-decoding.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions