fix(ci): treat API-error triage responses as failed runs in qwen-triage.yml - #10333
Merged
Conversation
The `Check triage response` step of qwen-triage.yml classified any
non-empty response as a successful triage. A model-layer API error
response ("[API Error: Connection error. ...]", 268 chars) is a
non-empty string, so run 33070765162 (triage for #10285) reported
success, posted nothing, and no retry or alert fired.
Strip the known rate-limit guidance suffixes, right-trim, and fail the
step when the response ends with the "[API Error: ...]" shape -- the
pattern qwen-code-pr-review.yml already uses for the same CLI output
behavior (the stream-json adapter appends the formatted error last).
This covers both bare error responses and errors appended after partial
output, while a legitimate summary that merely quotes an API error
mid-prose stays green. The existing failure surface then engages: red
run, "ended early" lifecycle comment, re-run path.
Adds executed-step tests in scripts/tests/qwen-triage-workflow.test.js
following the existing check-step harness: the verbatim 268-char
response, an appended-error response, and a quota error with its
guidance suffix must fail; a quoted-error summary, a normal response,
empty, and 'null' keep their existing behavior.
Fixes #10314
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
github-actions
Bot
added
the
review/self-reported
Collaborator
|
✅ Qwen Triage finished — CI landed green on ✅ Qwen Triage 已完成 —— |
Collaborator
|
Thanks for the PR! Template looks good ✓ Problem: observed, not theoretical. Run 33070765162 — a real Direction: aligned — this is the triage pipeline failing silently. Routing an API-error response into the existing failure surface (red run, "ended early" lifecycle comment, re-run path) is the right fix, and the issue was split out of the #10292 thread with exactly this scope ("one file, one condition"). Size: not core infrastructure — 24 added lines in Approach: the scope feels right. One step, one added condition, reusing the exact classifier Risk: no elevated risk signals (no high-risk paths matched). Moving on to code review. 🔍 中文说明感谢贡献！ 模板完整 ✓ 问题：已观测到，不是理论性的。运行 33070765162 —— 一次真实的 方向：对齐 —— 这是 triage 流水线在静默失败。把 API 错误响应导入既有的失败面（运行标红、生命周期评论变为 "ended early"、可走重跑路径）是正确的修法；该 issue 正是从 #10292 线程按此范围（"一个文件、一个条件"）拆出来的。 规模：不涉及核心基础设施 —— 方案：范围合理。一个步骤、一个新增条件，复用了 风险：无升级风险信号（未命中高风险路径）。进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
Collaborator
Code reviewMy independent proposal before reading the diff: detect the failure in the Static read of the bash is clean: the parameter expansions are correctly quoted (safe under the step's No Critical findings, no convention violations. The executed-step test follows the existing pattern for this step in the same file and genuinely pins the change: it extracts the real step body from the workflow YAML and runs it under bash against the verbatim 268-char incident payload, so removing the new block would turn the suite red. Test evidenceUnattended CI run — per the gate rules I do not build or execute PR code; the evidence below is this commit's own CI, fetched via the API. The executed-step test is the behavioral oracle here (it runs the real step script, not a copy), so there is no separate Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。 Remaining in-flight items ( 中文说明代码审查我在读 diff 之前的独立方案：在 bash 静态审读无误：参数展开引号正确（在步骤的 无阻断性问题，无规范违规。执行式步骤测试沿用了同文件中该步骤的既有模式，并且真正钉住了改动：它从工作流 YAML 中提取真实步骤体并在 bash 下对逐字的 268 字符事故载荷运行，删掉新增代码块套件就会标红。 测试证据无人值守 CI 运行——按门禁规则，不构建或执行 PR 代码；以下证据为本提交自身的 CI，经 API 获取。执行式步骤测试就是行为 oracle（运行的是真实步骤脚本，而非副本），因此没有额外的 剩余在途任务（ — Qwen Code · qwen3.8-max Reviewed at |
Collaborator
|
Confidence: 5/5 — clean gate, minimal diff, executed test replaying the verbatim production payload; approval deferred only until CI lands green on this commit. Stepping back: this is exactly the fix the incident called for. A triage run that answers with a bare API error was passing its own success check — the worst failure mode for a bot pipeline, because nothing alerts. The fix adds one condition to one step, reuses the classifier the sibling PR-review workflow already ships, and the executed-step test replays the real 268-char response from run 33070765162 — which I verified existed and concluded CI on — Qwen Code · qwen3.8-max Reviewed at |
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qwen-code-ci-bot
approved these changes
Aug 27, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
Collaborator
|
Qwen Code review did not complete successfully. The review pipeline failed before a review could be posted. A transient error is retried automatically; if you are seeing this, retry with |
yiliang114
enabled auto-merge
August 28, 2026 08:34
chiga0
approved these changes
Aug 28, 2026
chiga0
left a comment
chiga0
left a comment
Collaborator
There was a problem hiding this comment.
No blocking findings.
Checked:
- Shell logic in
.github/workflows/qwen-triage.yml— suffix stripping loop (${BODY%"$S"}), POSIX right-trim (${BODY%"${BODY##*[![:space:]]}"}")), andcaseglob pattern (*"[API Error: "*"]") are all correct and cover the stated failure modes. Verified by reading both ends of the contract: the same three known guidance suffixes and the same trailing-]match pattern asqwen-code-pr-review.yml(lines visible in cross-file context fetch). - Test in
scripts/tests/qwen-triage-workflow.test.js— execution-derived: extracts the live step body from the YAML viastep('Check triage response')and runs it underbash --noprofile --norc -eo pipefail, matching GitHub's exact shell flags. Covers: bare 268-char connection error, partial output + appended error, quota error + guidance suffix (all must fail); mid-prose API-error quote, normal response, empty,null(all keep existing behavior). - Cross-file consistency with
qwen-code-pr-review.yml: suffix list andcasepattern are an exact copy; the quota-specific branch inpr-review.ymlis correctly omitted since the triage context does not need it.
Unreviewed dimensions (disclosure):
review-prCI job:conclusion=failure— step "Run review" wascancelled(infrastructure issue; the bot still posted an approval via a separate mechanism). Unrelated to this code change.Test (macos-latest),Test (windows-latest),build-cli,Integration Tests: allSKIPPED— expected, since the diff touches only a CI workflow step body and a JS executed-step test; neither is built or run on those platforms.
Reviewed with AI assistance.
yiliang114
commented
Aug 28, 2026
yiliang114
left a comment
yiliang114
left a comment
Collaborator
Author
There was a problem hiding this comment.
Reviewed at 7d2b2b7 — no findings. Verified the detection order is right: strip the three known rate-limit suffixes, right-trim, then fail when the remainder ends with the [API Error: …] shape. The case pattern matches across newlines, so multi-line errors are covered; a real summary that merely quotes an API error mid-prose keeps writing after the closing bracket and stays green, which the tests pin alongside the verbatim 268-char string from run 33070765162, the partial-output-plus-error and quota-guidance shapes, and the unchanged empty/null behavior. Same pattern as qwen-code-pr-review.yml, as claimed.
Self-PR: GitHub blocks self-approval, so this is a comment review; merge approvals need to come from other maintainers.
Collaborator
|
Released in v0.22.3. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Makes the
Check triage responsestep ofqwen-triage.ymlfail when the triage action's response is a model-layer API error instead of a triage: it strips the known rate-limit guidance suffixes, right-trims, then fails the step when what remains ends with the[API Error: ...]shape. One step, one added condition; no other step is touched.Why it's needed
The step currently classifies any non-empty response as a successful triage. An API error response such as
[API Error: Connection error. (cause: connect ETIMEDOUT ...)]is a non-empty string, so it passed the check: run 33070765162 (triage for #10285) returned a 268-char connection error, the run reported success, nothing was posted, and no retry or alert fired. This PR routes that response shape into the existing failure surface: a red run, the "ended early" lifecycle comment, and the re-run path. It reuses the patternqwen-code-pr-review.ymlalready ships for the same CLI output behavior (the stream-json adapter appends the formatted error last, optionally followed by rate-limit guidance), so an error appended after partial output is caught too while a legitimate summary that merely quotes an API error mid-prose stays green.Reviewer Test Plan
How to verify
main: extract theCheck triage responsestep body and run it under GitHub's exact shell flags (bash --noprofile --norc -eo pipefail) withRESPONSEset to the verbatim 268-char error string from run 33070765162. Observed:Triage response received (268 chars)., exit 0 — matching the production log line.::error title=Triage aborted by an API error::...and exits 1; a normal response still printsTriage response received (... chars).and exits 0.npx vitest run --config ./scripts/tests/vitest.config.ts scripts/tests/qwen-triage-workflow.test.js. The new test covers: the verbatim 268-char response, an error appended after partial output, and a quota error ending in the rate-limit guidance suffix (all must fail the step); a summary that merely quotes an API error mid-prose, a normal response, empty, andnull(all keep their existing behavior).Evidence (Before & After)
Before (step body from
main, run with the real 268-char response):Triage response received (268 chars)., exit 0 — identical to the production log of run 33070765162. After (this branch, same input):::error title=Triage aborted by an API error::The model API returned an error instead of a triage, so no triage was posted. This is infrastructure, not this issue -- re-run the failed job., exit 1.Tested on
Environment (optional)
Vitest executed-step tests; the step script runs under GitHub's exact shell flags (
bash --noprofile --norc -eo pipefail).Risk & Scope
[API Error: ...](nothing written after it) would be failed and need a re-run; this is the same tradeoffqwen-code-pr-review.ymlalready ships with this pattern (itssuccess_mentions_api_errorcase covers the mid-prose quote).Linked Issues
Fixes #10314
中文说明
本 PR 做了什么
让
qwen-triage.yml的Check triage response步骤在 triage action 的响应是模型层 API 错误而非 triage 结果时判定失败：先剥离已知的限流提示后缀、右去空白，当剩余内容以[API Error: ...]形状结尾时使该步骤失败。只改一个步骤、只加一个判定条件，不动其他步骤。为什么需要
该步骤目前把任何非空响应都判为成功的 triage。像
[API Error: Connection error. (cause: connect ETIMEDOUT ...)]这样的 API 错误响应也是非空字符串，因此能通过检查：运行 33070765162（#10285 的 triage）返回了 268 字符的连接错误，运行却报告成功、什么都没发，也没有触发任何重试或告警。本 PR 把这种响应形状导入既有的失败面：运行标红、生命周期评论变为 "ended early"、可走重跑路径。它复用了qwen-code-pr-review.yml针对同一 CLI 输出行为已经在用的模式（stream-json adapter 会把格式化后的错误追加在最后，可能再跟一段限流提示），因此部分输出后追加错误的情况也能被捕获，而正文中仅仅引用 API 错误的正常 summary 仍判为成功。审阅者测试方案
如何验证
main上复现旧行为：提取Check triage response步骤体，用 GitHub 的原始 shell 参数（bash --noprofile --norc -eo pipefail）运行，并把RESPONSE设为运行 33070765162 中逐字的 268 字符错误字符串。观察到：Triage response received (268 chars).，退出码 0 —— 与生产日志行一致。::error title=Triage aborted by an API error::...且退出码为 1；正常响应仍输出Triage response received (... chars).且退出码为 0。npx vitest run --config ./scripts/tests/vitest.config.ts scripts/tests/qwen-triage-workflow.test.js。新测试覆盖：逐字的 268 字符响应、部分输出后追加错误、以及以限流提示后缀结尾的配额错误（三者都必须使步骤失败）；仅引用 API 错误的 summary、正常响应、空响应与null（均保持既有行为）。证据（修改前后）
修改前（
main的步骤体，输入真实的 268 字符响应）：Triage response received (268 chars).，退出码 0 —— 与运行 33070765162 的生产日志完全一致。修改后（本分支，同样输入）：::error title=Triage aborted by an API error::The model API returned an error instead of a triage, so no triage was posted. This is infrastructure, not this issue -- re-run the failed job.，退出码 1。测试环境
环境（可选）
Vitest 执行式步骤测试；步骤脚本以 GitHub 的原始 shell 参数（
bash --noprofile --norc -eo pipefail）运行。风险与范围
[API Error: ...]结尾（其后没有其他内容）的正常 triage summary 会被判失败、需要重跑；这与qwen-code-pr-review.yml用同一模式已经承担的取舍一致（其success_mentions_api_error用例覆盖正文中间引用的情况）。关联 Issue
Fixes #10314