Add a handler for Qwen3.5's XML tool-call format

[ajvikram]

What this adds

Qwen35FCHandler, plus registry entries for Qwen/Qwen3.5-2B-FC and Qwen/Qwen3.5-4B-FC as self-hosted function-calling models.

Why

Qwen3.5 changed the tool-call wire format. Qwen3 emits JSON inside <tool_call> tags, which QwenFCHandler parses. Qwen3.5 uses the XML style shared with Qwen3-Coder:

<tool_call>
<function=get_weather>
<parameter=city>
Berlin
</parameter>
</function>
</tool_call>

Running a Qwen3.5 model through QwenFCHandler yields no parsed calls, so the model scores near zero for a reason that has nothing to do with its ability.

How it works

1. Prompts are rendered with the model's own chat_template, through the tokenizer OSSHandler already loads, with tools converted by convert_to_tool to OpenAI style. The prompt is byte-identical to what the model was trained on.
2. Parsing extracts the XML calls, then re-types every parameter from the function spec of the current test entry, since XML parameter values are untyped text (10 must become an int when the schema says integer, ["a","b"] a list, and a ZIP code must stay a string). The spec index is thread-local because the handler instance is shared by the generation thread pool.
3. Output is stored as the typed Qwen3-style JSON <tool_call> form, so QwenFCHandler.decode_ast, decode_execute and the AST checker are reused unchanged rather than duplicated.

Thinking is left at the template default, so the generation prompt ends with an empty <think></think> block, which is how these checkpoints are meant to be served for tool use.

Verification

Both entries resolve to the new handler:

Qwen/Qwen3.5-2B-FC -> Qwen35FCHandler | model_name Qwen/Qwen3.5-2B | fc True | underscore_to_dot True
Qwen/Qwen3.5-4B-FC -> Qwen35FCHandler | model_name Qwen/Qwen3.5-4B | fc True | underscore_to_dot True

Scores for Qwen/Qwen3.5-2B-FC from a local run with this handler (vLLM, bf16, temperature 0.001, 128k serving context; the 32k default rejects about forty multi_turn_long_context entries):

Group	Accuracy
Overall	33.85%
Non-live AST	60.08%
Live	71.13%
Multi-turn	17.75%
Memory	30.97%
Web search	9.50%

Run with:

vllm serve Qwen/Qwen3.5-2B --dtype bfloat16 --max-model-len 131072
bfcl generate --model Qwen/Qwen3.5-2B-FC --test-category all_scoring --skip-server-setup --num-threads 16
bfcl evaluate --model Qwen/Qwen3.5-2B-FC --test-category all_scoring

Note that vLLM 0.17 or newer is needed for the Qwen3.5 architecture.

Happy to adjust naming or split this differently if you would prefer.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

The new handler’s dtype parameter is currently silently ignored due to base-class initialization behavior, which can lead to incorrect local server dtype selection.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds first-class support for Qwen3.5 self-hosted function-calling models by introducing a dedicated handler that can parse Qwen3.5’s XML-ish tool-call output and convert it into the existing Qwen3 JSON <tool_call>{...}</tool_call> form reused by current decoders/checkers.

Changes:

• Introduces Qwen35FCHandler to render prompts via the model’s chat_template and parse/coerce XML tool-call parameters into typed JSON tool calls.
• Registers Qwen/Qwen3.5-2B-FC and Qwen/Qwen3.5-4B-FC in the model registry/config.
• Documents the new self-hosted Qwen3.5 FC entries in SUPPORTED_MODELS.md.

File summaries

File	Description
berkeley-function-call-leaderboard/SUPPORTED_MODELS.md	Adds documentation row for Qwen3.5 FC self-hosted entries.
berkeley-function-call-leaderboard/bfcl_eval/model_handler/local_inference/qwen3_5_fc.py	New handler for Qwen3.5 XML tool-call parsing + prompt rendering via chat_template.
berkeley-function-call-leaderboard/bfcl_eval/constants/supported_models.py	Adds Qwen3.5 FC model IDs to supported model list.
berkeley-function-call-leaderboard/bfcl_eval/constants/model_config.py	Wires new model IDs to Qwen35FCHandler and model metadata.

Review details

Suppressed comments (1)

berkeley-function-call-leaderboard/bfcl_eval/model_handler/local_inference/qwen3_5_fc.py:87

• The string-type check is redundant ("any" is included in the tuple and also checked again with t == "any"). This doesn't change behavior, but simplifying reduces cognitive overhead.

        if t in ("string", "str", "any", "") or t == "any":

• Files reviewed: 4/4 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[ajvikram]

Thanks for the review. All three points are addressed in the latest commit.

dtype is silently ignored — confirmed, and it is worth flagging where it comes from. QwenFCHandler.__init__ accepts dtype but calls super().__init__(model_name, temperature, registry_name, is_fc_model, **kwargs) without it, so it never reaches OSSHandler, which is where self.dtype is set and later used for the server's --dtype flag. My handler inherits that, so a non-default dtype would have been dropped when launching vLLM or SGLang.

I have set self.dtype = dtype in this handler with a comment explaining why. That keeps the fix inside the file this PR adds.

The same latent bug affects the existing Qwen/Qwen3-*-FC entries through QwenFCHandler. It is a one-line change there too. I left it out to keep this PR to a single concern, but I am happy to include it here or open a separate PR, whichever you prefer.

Duplicate "array" in _LIST_TYPES and the redundant t == "any" check are both removed.

[ajvikram]

@HuanzhiMao gentle ping whenever you have a moment.

This adds Qwen3.5 support, which isn't on main yet. Qwen3.5 emits tool calls in an XML format, so the existing Qwen handler parses nothing and the model scores near zero for reasons unrelated to its ability. The Copilot review findings, including the dtype one, are addressed.

It would also unblock #1319, a Qwen3.5-0.8B fine-tune that currently has results but no handler to produce them.

Happy to rebase or change anything you'd like.