fix(core): add modality metadata for qwen3.8-flash and qwen3.8-plus - #10195
Conversation
qwen3.8-flash and qwen3.8-plus were classified as text-only by defaultModalities() because MODALITY_PATTERNS only had qwen3.8-max. This caused read_file on images to route through the vision bridge (secondary model call) instead of native inline attachment. Also adds modalities metadata to the alibabaStandard preset for qwen3.6-plus and qwen3.7-plus, matching the token-plan preset. Fixes QwenLM#10194 Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Thanks for the PR, @yiliang114! The underlying problem is well established — #10194 carries authoritative catalog evidence and a before/after reproduction, and the root cause still holds on main (qwen3.8-flash/qwen3.8-plus fall through to the text-only catch-all, and the alibabaStandard preset has no modalities metadata). Before we can review the change itself, though, the PR description needs to follow the repository's pull request template.
The current body uses free-form headings (Summary, Problem, Changes, Verification), but none of the template's required sections are present:
## What this PR does## Why it's needed## Reviewer Test Plan(with### How to verify,### Evidence (Before & After), and the### Tested onOS table)## Risk & Scope## Linked Issues- the bilingual
<details>Chinese summary the template asks for
Please edit the PR body to follow the template: https://github.com/QwenLM/qwen-code/blob/main/.github/pull_request_template.md
The substance you've already written maps cleanly onto those sections — the linked issue already carries the reproduction and before/after evidence — so this is a reformat, not new work. Once the body follows the template, re-trigger triage (@qwen-code /triage) and we'll continue with the review.
中文说明
感谢提交 PR，@yiliang114！问题本身已经很扎实——#10194 提供了官方能力目录证据和 before/after 复现，并且根因在 main 上依然存在（qwen3.8-flash/qwen3.8-plus 落入 text-only 兜底规则，alibabaStandard preset 也完全没有 modalities 元数据）。不过在进入代码审查之前，PR 描述需要先按照仓库的 pull request 模板来填写。
当前正文使用的是自由格式的小节（Summary、Problem、Changes、Verification），模板要求的小节一个都没有：
## What this PR does## Why it's needed## Reviewer Test Plan（包含### How to verify、### Evidence (Before & After)以及### Tested on操作系统表格）## Risk & Scope## Linked Issues- 模板要求的双语
<details>中文说明
请按照模板编辑 PR 正文：https://github.com/QwenLM/qwen-code/blob/main/.github/pull_request_template.md
你已经写好的内容可以很自然地对应到这些小节——关联 issue 里已经有完整的复现和 before/after 证据——所以这只是重新排版，不需要新增内容。正文符合模板后，重新触发 triage（@qwen-code /triage），我们会继续审查。
— Qwen Code · qwen3.8-max
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline.
— qwen3.8-max via Qwen Code /review (v0.22.2)
…ties tests Address two CI bot suggestions on PR QwenLM#10195: 1. alibaba-standard.test.ts: assert modalities inside generationConfig so removing the field from the provider config would fail the test. 2. modalityDefaults.test.ts: use toEqual({image:true,video:true}) instead of individual toBe(true) checks, pinning the exact shape and catching accidental extra modality keys. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
@qwen-code /triage |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 52 passed · 0 failed · 52 total Flakiness gate: ✅ 2 changed test file(s) x 5 identical rounds, no divergence 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：52 通过 · 0 失败 · 52 总计 抖动门：✅ 2 changed test file(s) x 5 identical rounds, no divergence Verification reportPR #10195 Deep Verification —
|
| Cell | Oracle | Base tip | PR head |
|---|---|---|---|
defaultModalities('qwen3.8-flash') |
return value | {} (text-only) |
{image:true, video:true} |
defaultModalities('qwen3.8-plus') |
return value | {} |
{image:true, video:true} |
siblings: -preview, dated -2026-08-01, -latest, QWEN3.8-FLASH, dashscope/…, qwen-oauth:… (6 cells) |
return value | {} ×6 |
{image,video} ×6 |
qwen3.8-max / qwen3.8-max-preview (controls) |
return value | {image:true} |
{image:true} (unchanged) |
qwen3.7-plus, qwen3.6-plus, qwen3.6-flash, qwen3-coder-plus, unknown (controls) |
return value | identical | identical |
isImageCapable({id:'qwen3.8-flash'}) — vision-bridge-service.ts:62 expression |
bool | false |
true |
isImageCapable({id:'qwen3.8-plus'}) |
bool | false |
true |
15 model IDs × 2 arms = 30 cell assertions, all as predicted (expected base reds encoded as expectations, so all count as passes). Witness: evidence/01-ab-modality-cells.png (raw cells as printed, logs/ab-harness.log).
Downstream chain verified by reading: defaultModalities() fills generationConfig.modalities in modelRegistry.resolveModelConfig (line 321), modelsConfig (441, 937) and modelConfigResolver (430); isImageCapable (vision-bridge-service.ts:62) and fileUtils.ts:1224 (read_file PDF/image handling) consume the result. qwen3.8-flash/qwen3.8-plus appear in no provider preset (grepped packages/core/src/providers), so defaultModalities() is the sole source for these IDs — the change is load-bearing for every setup path.
Secondary claim 1 — preset → install plan (both arms)
| Cell | Base tip | PR head |
|---|---|---|
buildInstallPlan(alibabaStandard, …['qwen3.6-plus','qwen3.7-plus','custom-model']) → generationConfig['qwen3.6-plus'] |
{extra_body, contextWindowSize} — no modalities |
adds modalities:{image,video} |
same for qwen3.7-plus |
no modalities | adds modalities:{image,video} |
generationConfig['custom-model'] |
undefined |
undefined (unchanged) |
getDefaultModelIds(alibabaStandard) |
identical | identical |
Same table, same witness (01-ab-modality-cells.png, sections C–D).
Corrections
Correction to the PR description (not a code-change request): the body states the alibabaStandard preset "lacked modalities metadata entirely, so Standard API-key users had no path to correct capability data." Tracing every read site shows this overstates the effect: the name patterns /^qwen3\.6-plus/ and /^qwen3\.7-plus/ already existed at base (they are context lines in the diff), and every consumer of modalities falls back to defaultModalities() when the persisted field is absent (vision-bridge-service.ts:62, modelRegistry.ts:321, modelsConfig.ts:441/937, modelConfigResolver.ts:430, via contentGeneratorConfig into fileUtils.ts:1224 and the generators). Section D of the A/B harness measured this directly: runtime modalities for a Standard-installed qwen3.6-plus are {image,video} on both arms. The preset hunk is therefore redundant defence / self-describing data (consistent with the token-plan preset), not a behavioural routing fix. It is harmless, and the strengthened test pins it — no action needed, but the motivation as written is inaccurate.
Findings
None. No code defect was produced by the evidence.
Mutation matrix (vacuity)
Each mutant was applied to a backup-guarded copy of the real source, the target suite run, and the file restored (git status clean after each). Every row is log-verified by summary-harness.mjs (logs/mutant-M*.log); witness evidence/02-mutation-matrix.png.
| # | Mutation | Expected catcher | Result |
|---|---|---|---|
| M1 | revert pattern hunk → base state | the 2 new regression tests | KILLED — 2 failed | 50 passed (52), both fail on expected {} to deeply equal { image: true, video: true } (behavioural mismatch, not setup breakage) |
| M2 | qwen3.8-max → {} (positive control, same file) |
existing max tests | KILLED — 2 existing tests fail, proving the vitest invocation collects this file |
| M3 | revert preset modalities → base state | install-plan test (commit 2) | KILLED — 1 failed | 7 passed (8), failure diff names the missing modalities block |
| M4 | drop enableThinking on deepseek-v4-pro (positive control, same file) |
DeepSeek test | KILLED — a different existing test fails |
| M5 | add extra audio:true to flash entry |
toEqual exact-shape (commit 2's claim) |
KILLED — diff cites + "audio": true; old-style toBe(true) checks would have survived this |
No survivors. The positive controls landed in the same files as their mutants, so the kills are attributable to the suites, not to a harness accident.
Targeted gates (head, unmutated)
| Gate | Scope | Result |
|---|---|---|
| G1 | modalityDefaults.test.ts + alibaba-standard.test.ts |
60/60 (52 + 8 — matches the PR's claimed 52/52) |
| G2 | entire src/providers/ dir (blast radius) |
172/172, 16 files |
| G3 | consumer suites: modelConfigResolver.test.ts, src/services/visionBridge/, dashscope.test.ts |
312/312, 5 files |
Zero failures at head, so no base-side failure attribution was needed.
Not covered
- Per-commit attribution — the depth-2 checkout reaches only the merge commit, base tip, and PR head tip (
git rev-list HEAD^1..HEAD^2returns 1 commit vs the snapshot's 2). The aggregateHEAD^1..HEADdiff is what was verified; commit 2's claims were nevertheless exercised individually (M3, M5 target exactly its changes). - External capability facts — whether
qwen3.8-flash/qwen3.8-plustruly accept image+video input on the DashScope API is model-metadata asserted by the author; the sandbox has no network or credentials to verify it against the vendor. Same for the pre-existing asymmetry thatqwen3.8-maxis image-only inMODALITY_PATTERNSbut{image,video}in the token-plan preset — pre-existing at base, untouched by this PR. - Sibling model
qwen3.6-flash(listed in token-plan with no modalities; name table classifies it text-only) — same bug class, pre-existing, unchanged by this PR; whether it is actually multimodal is external data. - Full end-to-end read_file routing with a real image and a real model call — no credentials in the sandbox. The A/B reproduces the routing decision (the exact
isImageCapableexpression and the modalities-fill chain), i.e. the shape of the mechanism, not the wire-level cause on a live API. - Typecheck / lint / format — covered by the PR's own CI; not re-run here (vitest transformed all changed TS cleanly during the gates).
Methodology
Environment: CI verify container (node:22-bookworm, node v22.23.2), merge-ref checkout at depth 2 (HEAD = merge 2c1e6765fd, HEAD^1 = base tip, HEAD^2 = PR head), npm ci + npm run build pre-run. A/B control: scratch worktree at a82a11a0 under tmp/base-tree; the four code units under test (modalityDefaults.ts ×2 arms, provider-config.ts + alibaba-standard.ts ×2 arms) were esbuild-bundled per arm from source, so no workspace symlink could leak head code into the base arm — the only shared dependency resolution was third-party (packages/core/node_modules symlinked into the base tree for the OpenTelemetry/ajv nested install; verified it contains zero @qwen-code/* entries, and the PR touches no package.json/lockfile). Control purity proven by diffing the two modality bundles (differ by exactly the PR hunk; logs/control-bundle-diff.log). Harnesses (ab-harness.mjs, summary-harness.mjs) encode every expectation — predicted base failures count as passes; fail counts only unexpected outcomes. Mutation round edited real source files with prior backups and restored after each run (worktree verified clean). Raw per-run logs live in logs/; builds in builds/.
Flakiness gate log
rounds=5 files=2 skipped=0
file packages/core/src/core/modalityDefaults.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/modalityDefaults.test.ts
file packages/core/src/providers/__tests__/presets/alibaba-standard.test.ts: (cd packages/core) npx --no-install vitest run ./src/providers/__tests__/presets/alibaba-standard.test.ts
per-file results (P=pass F=fail I=infra-exit, one letter per run):
packages/core/src/core/modalityDefaults.test.ts: PPPPP
packages/core/src/providers/__tests__/presets/alibaba-standard.test.ts: PPPPP
verdict: pass
summary: 2 changed test file(s) x 5 identical rounds, no divergence
--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/core/src/core/modalityDefaults.test.ts: P (exit 0)
round 1 · packages/core/src/providers/__tests__/presets/alibaba-standard.test.ts: P (exit 0)
round 2 · packages/core/src/core/modalityDefaults.test.ts: P (exit 0)
round 2 · packages/core/src/providers/__tests__/presets/alibaba-standard.test.ts: P (exit 0)
round 3 · packages/core/src/core/modalityDefaults.test.ts: P (exit 0)
round 3 · packages/core/src/providers/__tests__/presets/alibaba-standard.test.ts: P (exit 0)
round 4 · packages/core/src/core/modalityDefaults.test.ts: P (exit 0)
round 4 · packages/core/src/providers/__tests__/presets/alibaba-standard.test.ts: P (exit 0)
round 5 · packages/core/src/core/modalityDefaults.test.ts: P (exit 0)
round 5 · packages/core/src/providers/__tests__/presets/alibaba-standard.test.ts: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
|
Thanks for the PR, and for addressing both Template: the body is still free-form — the previous pass stopped at this exact gate. Since the author is a maintainer of this repo and re-triggered triage with all the substance in place (what/why/verification in the body, full reproduction in the linked issue), I'm not gating on heading formatting a second time and will proceed. Flagging it so this is a visible, deliberate call rather than a silent skip. Problem: real and well evidenced. #10194 carries authoritative catalog output ( Direction: aligned. This is the same reactive catch-up the modality table has always done (per the issue's own history), and it fixes a silent quality + extra-billing regression for Standard API-key users. Size: core paths touched ( Approach: minimal — exactly the issue's suggested fixes 1 and 2, with the larger items (catalog-derived modalities, pinned user config, Risk: no elevated risk signals (no high-risk path matches). Moving on to code review. 🔍 中文说明感谢提交 PR，也感谢在新提交里处理了两条 模板：正文仍是自由格式——上一轮 triage 正是停在这一步。鉴于作者是仓库维护者，且重新触发 triage 时实质内容都已齐备（正文说明了做什么/为什么/如何验证，关联 issue 里有完整复现），不再第二次因标题格式拦截，直接进入审查。此处显式标注这一决定，而非悄悄跳过。 问题：真实存在且证据充分。#10194 提供了官方能力目录输出（ 方向：对齐。这与 modality 表一直以来的追赶式更新一致（见 issue 中的梳理），修复的是 Standard API-key 用户被静默降级（画质损失 + 额外计费）的问题。 规模：触及核心路径（ 方案：最小化——正是 issue 建议的修复 1、2 两项，更大的议题（从目录派生 modality、固定用户配置、 风险：无升级风险信号（未命中高风险路径）。 进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
Code reviewIndependent proposal first: for this problem I'd have done exactly what this PR does — two Details I verified while reading the diff:
No blockers, no convention violations. Test evidence — this PR's own CIEvery vitest suite in the ubuntu job passed, including both touched files — Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。 Not verified: the end-to-end behavior — that 中文说明代码审查先说独立方案：换作我来修，做法与本 PR 完全一致——在 读 diff 时逐项确认：
无阻塞项，无规范违反。 测试证据——本 PR 自己的 CIubuntu job 中所有 vitest 套件全部通过，包括两个被修改的文件—— （CI 明细见上方英文表格。） 未验证：端到端行为——选中 — Qwen Code · qwen3.8-max Reviewed at |
|
Confidence: 4/5 — clean, minimal, well-evidenced fix; the only blemish is the red ubuntu check, which the evidence pins on the test-efficacy harness rather than this PR. Stepping back: this is exactly the kind of PR the gate should wave through. The problem was observed and reproduced with authoritative catalog evidence (#10194), the root cause is verified still present on Approving on the reviewed commit. ✅ 中文说明置信度：4/5 —— 干净、最小化、证据充分的修复；唯一的瑕疵是红色的 ubuntu 检查，而证据表明那是 test-efficacy 校验器自身的问题，与本 PR 无关。 退一步看：这正是门禁应当放行的那类 PR。问题是真实观测到的，#10194 提供了官方目录证据和复现；根因已确认在 已按所审查的提交批准。✅ — Qwen Code · qwen3.8-max Reviewed at |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline.
Not explored to full depth (tool budget reached): "agent 6b": run the two test files to confirm green — the review worktree has no node_modules or packages/core/dist (verified), and npm ci + build exceeds the tool bu….
— qwen3.8-max via Qwen Code /review (v0.22.2)
| contextWindowSize: 1000000, | ||
| modalities: { image: true, video: true }, |
There was a problem hiding this comment.
[Suggestion] R1-1: still stands — the pin delivered in 16ad5b8 is narrower than the original suggestion asked, in two ways.
qwen3.7-plus is still not pinned at all: the install-plan fixture is modelIds: ['qwen3.6-plus', 'custom-model'], and no other test asserts this preset's qwen3.7-plus modalities (the similarly-named pin in alibaba-token-plan.test.ts binds the token-plan preset's separate spec). Removing modalities from the qwen3.7-plus entry in alibaba-standard.ts leaves the whole suite green. And because this qwen3.6-plus pin sits inside a toMatchObject assertion — a recursive subset matcher — adding an extra key to the preset entry's modalities survives it too, whereas the original suggestion's example (and the sibling DeepSeek test below) used exact-shape toEqual.
Both escape routes were verified with mutant runs in a scratch tree at this commit:
baseline (delivered test): 8 passed
mutant pdf:true on qwen3.6-plus: 8 passed <- escapes the delivered pin
mutant modalities removed from qwen3.7-plus: 78 passed <- escapes the delivered pin
exact-shape fix (toEqual, both entries): both mutants fail
Suggested fix: append qwen3.7-plus to the install-plan fixture and pin both entries exact-shape (appending keeps the existing models?.[0]/models?.[1] assertions unchanged):
modelIds: ['qwen3.6-plus', 'custom-model', 'qwen3.7-plus'],
// ...
expect(models?.[0]?.generationConfig?.modalities).toEqual({
image: true,
video: true,
});
expect(models?.[2]?.generationConfig?.modalities).toEqual({
image: true,
video: true,
});Fix witness: this same 'creates an install plan with editable models' test must go red when modalities is removed from either preset entry, and when an extra modality key is added — please confirm with a mutant run (remove the field, run this test, watch it fail).
— qwen3.8-max via Qwen Code /review (v0.22.2)
…fix scoped The qwen3.6/3.7-plus preset entries are already covered by MODALITY_PATTERNS via the defaultModalities() fallback, so explicit preset metadata is redundant for fixing QwenLM#10194.
chiga0
left a comment
chiga0
left a comment
There was a problem hiding this comment.
No blocking findings. Approved.
Scope: modalityDefaults.ts + modalityDefaults.test.ts only. The alibaba-standard.ts changes from earlier commits were intentionally dropped (commit 56db59e0) — current diff is 2 files, 18 lines net.
What I checked:
- Pattern correctness:
^qwen3\.8-flashand^qwen3\.8-plusare properly anchored (^) and escaped (\.), consistent with all siblings inMODALITY_PATTERNS. Both are inserted before the existingqwen3\.8-maxentry — ordering is correct. - normalize() contract:
normalize()lowercases and strips provider prefixes/suffixes before matching, soQwen3.8-Flash,bailian/qwen3.8-flash,qwen3.8-flash:latestall resolve toqwen3.8-flashand match. The existingqwen3.8-max-previewtest confirms this path. - Test validity: Both new tests use
toEqual({ image: true, video: true })— exact match. Adding any extra key (e.g.pdf: true) would fail both tests. Absence is asserted implicitly. - catch-all position: The
[/^qwen/, {}]text-only catch-all is downstream of the new entries; new patterns win by position.
Cross-check against prior reviews:
| Prior finding | Status |
|---|---|
| R1-2 (absent-modality assertion in new tests) | Resolved — current tests use toEqual; a mutant adding pdf: true would now fail |
R1-1/R2 (alibaba-standard.ts qwen3.7-plus not pinned) |
Moot — those changes were scoped out in the final commit; the file is not in the diff |
CI at head 56db59e0:
| Check | Result |
|---|---|
| Test (ubuntu-latest, Node 22.x) | in_progress at review time |
| Test (windows-latest) | SKIPPED |
| Test (macos-latest) | SKIPPED |
| Integration Tests | SKIPPED |
| Desktop Shell (ubuntu, windows) | ✅ success |
No platform-specific logic in this diff; SKIPPED test platforms are not a coverage gap here. Ubuntu test was still running — if it comes back red, this approval should be revisited.
Reviewed with AI assistance.
|
Released in v0.22.3. |
…-max Resolve five conflicts; main had superseded most of this branch: - modalityDefaults.ts (+ its test): keep main's newer classification — qwen3.8-max is image-only there (QwenLM#10195/QwenLM#10278), not image+video. - alibaba-token-plan.ts: keep this branch's deepseek-v4-flash entry (the part main never landed) and drop its duplicate qwen3.8-max record; main's later QwenLM#9383 entry, with thinkingMandatory, is authoritative. The matching branch assertion is dropped with it. - vscode-ide-companion subscriptionPlanDefinitions.{ts,test.ts}: take main's version, which now derives the plan list from the shared preset instead of the hand-kept copy this branch was updating. core: alibaba-token-plan + modalityDefaults 62 passed; vscode-ide-companion subscriptionPlanDefinitions 3 passed.
5 participants
Summary
Add
qwen3.8-flashandqwen3.8-plustoMODALITY_PATTERNSwith{ image: true, video: true }.Problem
qwen3.8-flashwas classified as text-only bydefaultModalities()becauseMODALITY_PATTERNSonly had an entry forqwen3.8-max. This causedread_fileon images to silently route through the vision bridge (a second, separately billed model call) instead of native inline attachment.Fixes #10194
Changes
packages/core/src/core/modalityDefaults.ts: Addqwen3.8-flashandqwen3.8-plusentries before theqwen3.8-maxentry inMODALITY_PATTERNSpackages/core/src/core/modalityDefaults.test.ts: Add regression tests forqwen3.8-flashandqwen3.8-plusVerification
modalityDefaults.test.ts(including 2 new regression tests)