fix(ci): make shared ECS Vitest concurrency tunable - #10667
Merged
Conversation
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Collaborator
|
|
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Collaborator
|
Thanks for the PR! Template looks good ✓ Problem: observed, with solid evidence. Linked issue #10665 tracks main CI run 33452224969, and the PR's analysis of that run is concrete — unrelated suites failing together, normally fast tests taking 15–60 s, the shell classifier burning 1,856 ms of process CPU against its 1 s budget, while disk/inode/memory samples stayed healthy. Open umbrella issue #10490 describes the same non-deterministic failure shape on shared runners. Direction: aligned. This is the repo's own CI stability on the shared 128-core ECS host — fewer false-red main runs, no product surface touched. The follow-up commit extending the cap to release.yml's Size: not applicable — no core package paths. The diff is 7 lines each in Approach: sound, and simpler than the obvious alternative. I checked the mechanics against Vitest 3.2.7 (the locked version): Risk: no elevated risk signals (no high-risk paths touched). Moving on to code review. 🔍 中文说明感谢贡献！ 模板完整 ✓ 问题：真实观测到的问题，证据充分。关联 issue #10665 追踪 main CI run 33452224969，PR 对该运行的分析具体——互不相关的测试套件同时失败、原本很快的测试耗时 15–60 秒、shell classifier 在 1 秒预算下消耗 1,856 ms 进程 CPU，而磁盘/inode/内存采样均正常。未关闭的 #10490 也描述了共享 runner 上同类的非确定性失败。 方向：对齐。这是仓库自身在共享 128 核 ECS 宿主上的 CI 稳定性——减少 main 的误报失败，不触及产品面。后续 commit 把限额扩展到 release.yml 的 规模：不适用——未触及核心包路径。diff 为 方案：合理，且比显然的替代方案更简单。已对照锁定的 Vitest 3.2.7 源码核实机制：这些环境变量在 vitest 的 风险：无升级风险信号（未触及高风险路径）。 进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
Collaborator
Code reviewIndependent baseline first: given the repo already detects ECS runners inside its vitest configs ( I verified the mechanics against the Vitest 3.2.7 source (the version locked in
The second commit extends the same block to release.yml's No blockers, no convention violations. Testing evidenceUnattended CI run — I read this PR's own CI checks via the API at the reviewed commit; I did not build or execute PR code. Fetched once, no polling: the unit-test job below was still running at fetch time, and the The PR's new pin tests run inside the Ubuntu unit-test job ( Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。 What this evidence can and cannot settle: the pin tests prove the YAML wiring, and the worker-limit semantics are verified above against the locked Vitest source. Whether contention failures on the ECS host actually decrease is only observable post-merge (watch the #10490 / #10665 failure class) — no sandboxed lane can reproduce 32-runner oversubscription, so there is no verification command that would settle it beforehand. 中文说明代码审查：先说独立基线——仓库的 vitest 配置里已有 ECS runner 探测（ 机制已对照锁定的 Vitest 3.2.7 源码核实（而非仅看文档）： 第二个 commit 把同样的块扩展到 release.yml 的 无阻塞项，无规范违反。 测试证据：无人值守 CI 运行——我通过 API 读取本 PR 自身在受审提交上的 CI 检查，未构建或执行 PR 代码。一次性拉取、不轮询：下表中的单测任务拉取时仍在运行， 本 PR 的新钉住测试在 Ubuntu 单测任务内运行（ 该证据能证明与不能证明的：钉住测试证明 YAML 接线正确，worker 上限语义已对照锁定的 Vitest 源码核实。ECS 宿主上的竞争失败是否真正减少，只能在合并后观察（关注 #10490 / #10665 一类失败）——没有任何沙箱通道能复现 32 个 runner 的过载，因此不存在可提前定论的验证命令。 — Qwen Code · qwen3.8-max Reviewed at |
Collaborator
|
Confidence: 4/5 — clean across every stage; the only nits are a stale sentence in the PR body and release.yml's nearly-exhausted size allowance, neither blocking. This one is what a good CI fix looks like. The problem is observed, not hypothesized — a specific red main run with symptoms that all point at CPU oversubscription, and the arithmetic (32 registrations × 25% of 128 cores ≈ 32 workers per job, 448 workers on 14 busy runners) checks out. The fix is the smallest instrument that reaches every vitest suite: one env block per test entry point, gated on the runner-name prefix this repo already uses for ECS detection, verified against the locked Vitest 3.2.7 source rather than assumed. My independent baseline — capping inside each workspace's vitest config — would have been more edits and still missed suites with no worker settings; the PR's path is the better one. The release.yml extension in the second commit earns its place: that If I'm maintaining this in six months, the pin tests keep the YAML blocks from drifting silently, and the env approach means no config surgery when the runner fleet changes — only the CI on the reviewed commit is not settled yet — the main 中文说明置信度：4/5——各阶段都干净；仅有的小问题是 PR 正文有一句过期描述、以及 release.yml 的体积余量接近用尽，均不阻塞。 这是一次规范的 CI 修复的样子。问题是观测到的，不是假设——具体的一次 main 红跑，症状全部指向 CPU 过载，且算术成立（32 个注册 × 128 核的 25% ≈ 每 job 32 个 worker，14 个忙碌 runner 上约 448 个 worker）。修复用的是能触达每个 vitest 套件的最小手段：每个测试入口一个 env 块，按本仓库既有的 半年后维护它时：钉住测试保证 YAML 块不会悄悄漂移；env 方案意味着 runner 集群变化时无需动配置，只有 受审提交上的 CI 尚未落定——主 — Qwen Code · qwen3.8-max Reviewed at |
yiliang114
enabled auto-merge
September 1, 2026 02:44
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
yiliang114
changed the title
This was referenced Sep 1, 2026
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/ci.yml: actionlint embedded-shell source mapping is not yet supported — not linted; the executable-script lint — .github/workflows/release.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
已审查——无阻断问题。 建议见行内评论。
未检查（工具限制，非阻断）：the executable-script lint — .github/workflows/ci.yml: actionlint embedded-shell source mapping is not yet supported — not linted; the executable-script lint — .github/workflows/release.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max via Qwen Code /review (v0.22.3)
| - name: 'Run Workspace Tests' | ||
| env: | ||
| # Match the tunable per-process bound used by the main CI gate. | ||
| VITEST_MAX_THREADS: "${{ startsWith(runner.name, 'ecs-qwen-') && (vars.QWEN_CI_VITEST_MAX_WORKERS || '4') || '' }}" |
Collaborator
There was a problem hiding this comment.
[Suggestion] R1-1: The Vitest worker bound ships as two byte-identical copies — this block in release.yml and the one in the ci.yml test job — and each copy is pinned only against its own inline test literal. Nothing relates the two copies, so the pin suite structurally cannot catch them drifting apart, even though this test family's own pattern is mutual equivalence pinning (the TMPDIR routing-block pin in scripts/tests/no-ak-integration-ci.test.js). If someone later tunes the bound in ci.yml — the fleet prefix, the '4' fallback, or a pool var — and updates only scripts/tests/no-ak-integration-ci.test.js, the release.yml copy and its pin literal in scripts/tests/package-scripts.test.js stay stale: both suites pass green while the release 'Run Workspace Tests' step silently runs with a different bound than the CI gate, indefinitely. One logical value, four edit sites.
Witness:
[probe] Mutation in scratch tree: ci.yml fallback '4' -> '8' with only the no-ak pin literal updated (release.yml and its pin left stale) — both pin suites still pass ('Test Files 2 passed (2)') while the two workflows disagree; adding one equivalence assertion turns the same mutation red ('expected Set ... to have a size of 1 but got 2').
Extract the expected expression strings (or a small helper) into scripts/tests/workflow-helpers.js — already imported by both test files — so both pins draw on one literal, and/or add one equivalence assertion that the two env blocks are byte-identical, e.g.:
const ciBlock = vitestEnvBlock(ciWorkflow, 'test', 'Run tests and generate reports');
const releaseBlock = vitestEnvBlock(releaseWorkflow, 'quality', 'Run Workspace Tests');
expect(new Set([ciBlock, releaseBlock])).toHaveLength(1);Fix constraint: scripts/tests/no-ak-integration-ci.test.js already pins cross-lane byte-identity for the TMPDIR routing block via expect(new Set(routingBlocks)).toHaveLength(1) — the new equivalence pin should follow that same pattern. Fix witness: with the literals shared, mutating the single constant must turn both pin tests red together, and the equivalence assertion must turn red when exactly one workflow's block is mutated — remove the guard, run those tests, and confirm they red.
中文说明
Vitest worker 上限以两份逐字节相同的副本存在——release.yml 中的这个块与 ci.yml test 任务中的那个——每一份都只被各自测试文件里的内联字面量钉住。两份副本之间没有任何断言关联，因此钉住测试在结构上无法发现二者漂移，尽管这个测试系列自己的惯例恰恰是互相等价钉住（scripts/tests/no-ak-integration-ci.test.js 中对 TMPDIR 路由块的钉住）。若有人之后在 ci.yml 里调节上限——runner 前缀、'4' 回退值或某个池变量——且只更新 scripts/tests/no-ak-integration-ci.test.js，release.yml 的副本和 scripts/tests/package-scripts.test.js 里的字面量会保持过期：两个套件全绿，而 release 的 'Run Workspace Tests' 步骤却悄悄以与 CI 门禁不同的上限运行，且无限期持续。一个逻辑值，四处修改点。
证据：[探针] 在临时树中变异：将 ci.yml 回退值 '4' 改为 '8' 且只更新 no-ak 钉住字面量（release.yml 及其钉住保持过期）——两个钉住套件仍全部通过（'Test Files 2 passed (2)'），而两个工作流已经不一致；加入一条等价断言后，同一变异变红（'expected Set ... to have a size of 1 but got 2'）。
修复建议：将期望的表达式字符串（或一个小 helper）提取到 scripts/tests/workflow-helpers.js——两个测试文件都已引入它——让两处钉住共用同一字面量；和/或新增一条断言钉住两个 env 块逐字节相同。修复约束：scripts/tests/no-ak-integration-ci.test.js 已通过 expect(new Set(routingBlocks)).toHaveLength(1) 钉住 TMPDIR 路由块的跨支线逐字节一致——新等价钉住应沿用该模式。修复见证：共享字面量后，变异该常量必须让两个钉住测试同时变红；只修改其中一个工作流的块时，等价断言必须变红——请移除该防护、运行这些测试并确认变红。
— qwen3.8-max via Qwen Code /review (v0.22.3)
| USERPROFILE: '${{ runner.temp }}/qwen-ci-home' | ||
| # Bound each Vitest process on the shared ECS host. Operators can | ||
| # tune the maximum without changing this workflow. | ||
| VITEST_MAX_THREADS: "${{ startsWith(runner.name, 'ecs-qwen-') && (vars.QWEN_CI_VITEST_MAX_WORKERS || '4') || '' }}" |
Collaborator
There was a problem hiding this comment.
[Suggestion] R1-2: The bound re-derives 'this job landed on the shared self-hosted pool' from the runner-name prefix, introducing a second detection mechanism where the repo already standardised one for this exact problem class: runner.environment → RUNNER_ENVIRONMENT → vitest pool cap (integration-tests/vitest.config.ts:14; RUNNER_ENVIRONMENT: '${{ runner.environment }}' wiring at e2e.yml:184 and release.yml:456/462/539; release.yml itself gates steps on runner.environment == 'self-hosted' at lines 424/437). The repo already anticipates hosts that do not match this prefix — qwen-autofix.yml:846-856 accepts ecs-qwen-*|ecs-agent-* because 'future dedicated hosts may use ecs-agent'. On such a host this expression evaluates to '', Vitest falls back to core-count workers, the contention this PR fixes returns, and nothing goes red — both pin tests pin the gate's literal string, not its effect.
Witness:
witness: not run — the trigger is GitHub runner-fleet state (a self-hosted host not named ecs-qwen-* serving the ecs-qwen label) and GitHub's expression evaluation of runner.name cannot be stood up locally; mechanism verified by quoted cross-file citations (integration-tests/vitest.config.ts:14; e2e.yml:184; release.yml:456/462/539; qwen-autofix.yml:846-856).
Gate on the existing signal in both workflows (all four vars), so the bound stays co-extensive with label-based routing:
VITEST_MAX_THREADS: "${{ runner.environment == 'self-hosted' && (vars.QWEN_CI_VITEST_MAX_WORKERS || '4') || '' }}"For these two jobs self-hosted ⇔ ecs-qwen today, so behaviour is unchanged while the bound can no longer drift from routing.
Fix constraint: integration-tests/vitest.config.ts:14 — const isSelfHostedRunner = process.env['RUNNER_ENVIRONMENT'] === 'self-hosted'; — the unit-lane gate should stay co-extensive with this existing self-hosted detection so both lanes bind and unbind under one fleet change. Fix witness: the expected strings in scripts/tests/no-ak-integration-ci.test.js ('uses a tunable max for every Vitest pool on ECS') and scripts/tests/package-scripts.test.js must be updated to the new gate and go red if the gate is removed or reverted to the name prefix — remove the guard, run those tests, and confirm they red.
中文说明
该上限通过 runner 名前缀重新推导'任务落在了共享自托管池上'，引入了第二种探测机制，而仓库对同一类问题已有标准化信号：runner.environment → RUNNER_ENVIRONMENT → vitest 池上限（integration-tests/vitest.config.ts:14；e2e.yml:184 与 release.yml:456/462/539 的 RUNNER_ENVIRONMENT: '${{ runner.environment }}' 接线；release.yml 自身在 424/437 行就用 runner.environment == 'self-hosted' 做门禁）。仓库已经预期不匹配该前缀的主机——qwen-autofix.yml:846-856 同时接受 ecs-qwen-*|ecs-agent-*，因为'未来专用主机可能使用 ecs-agent'。在这类主机上，本表达式求值为 ''，Vitest 回退到按核数开 worker，本要修复的资源竞争随之复发，而且不会有任何测试变红——两个钉住测试钉的是门禁的字面字符串，不是它的效果。
证据：未执行——触发条件是 GitHub runner 集群状态（一台名字不是 ecs-qwen-* 的自托管主机承接 ecs-qwen 标签任务），GitHub 对 runner.name 的表达式求值无法在本地搭建；机制已通过跨文件引用核实（integration-tests/vitest.config.ts:14；e2e.yml:184；release.yml:456/462/539；qwen-autofix.yml:846-856）。
修复建议：两个工作流都改用既有信号做门禁（四个变量均如此），使上限与基于标签的路由保持同外延。对这两个任务而言，当下 self-hosted ⇔ ecs-qwen，行为不变，但上限不再可能与路由漂移。修复约束：integration-tests/vitest.config.ts:14 的 isSelfHostedRunner 检测——单测通道的门禁应与其保持同外延，让两个通道在同一次集群变更下同时生效/失效。修复见证：两个钉住测试（no-ak 的 'uses a tunable max for every Vitest pool on ECS' 与 package-scripts 中的断言）的期望字符串需更新为新的门禁；门禁被删除或退回名字前缀时必须变红——请移除该防护、运行这些测试并确认变红。
— qwen3.8-max via Qwen Code /review (v0.22.3)
| # tune the maximum without changing this workflow. | ||
| VITEST_MAX_THREADS: "${{ startsWith(runner.name, 'ecs-qwen-') && (vars.QWEN_CI_VITEST_MAX_WORKERS || '4') || '' }}" | ||
| VITEST_MIN_THREADS: "${{ startsWith(runner.name, 'ecs-qwen-') && '1' || '' }}" | ||
| VITEST_MAX_FORKS: "${{ startsWith(runner.name, 'ecs-qwen-') && (vars.QWEN_CI_VITEST_MAX_WORKERS || '4') || '' }}" |
Collaborator
There was a problem hiding this comment.
[Suggestion] R1-3: The tunable vars.QWEN_CI_VITEST_MAX_WORKERS is passed verbatim into VITEST_MAX_THREADS/VITEST_MAX_FORKS, which Vitest parses with Number.parseInt — so a percentage value, the exact vocabulary the repo's adjacent ECS bounds use (maxWorkers: '25%' in packages/core/vitest.config.ts and packages/cli/vitest.config.ts), is silently truncated to an absolute worker count and never reaches Vitest's '%' resolver. An operator tuning QWEN_CI_VITEST_MAX_WORKERS=25% to match the sibling mechanism gets parseInt('25%') → 25, so a 64-core host runs 25 workers per Vitest process instead of the intended 16 and the contention this PR relieves returns — while both pin suites stay green, because they pin the expression literal, not the value's semantics. Fractional values misparse too: '0.5' → 0 workers.
Witness:
[probe] vitest 3.2.7 resolveConfig on a 64-core host:
env=25% -> poolOptions.threads.maxThreads=25, poolOptions.forks.maxForks=25
(config path maxWorkers:'25%' resolves to 16)
env=0.5 -> maxThreads=0 (tinypool then collapses min to max=0)
env=4 -> 4
State in the workflow comment that the value must be a plain integer worker count, e.g. extend the existing comment with: 'Value must be a plain integer worker count; Vitest applies Number.parseInt to these env vars, so a % suffix is truncated to an absolute count.' Alternatively validate the variable against an integer pattern before export.
Fix constraint: vitest 3.2.7 env parsing — maxThreads: Number.parseInt(process.env.VITEST_MAX_THREADS) (node_modules/vitest/dist/chunks/coverage.DfSpMS-b.js:3723); resolveInlineWorkerOption's '%' branch is unreachable for env-sourced values, so a fix must not promise percentage semantics through this path.
中文说明
可调变量 vars.QWEN_CI_VITEST_MAX_WORKERS 被原样传入 VITEST_MAX_THREADS/VITEST_MAX_FORKS，而 Vitest 用 Number.parseInt 解析它们——于是百分比取值（仓库相邻 ECS 限制所使用的词汇：packages/core/vitest.config.ts 与 packages/cli/vitest.config.ts 的 maxWorkers: '25%'）会被静默截断成绝对 worker 数，永远到不了 Vitest 的 '%' 解析分支。运维若为对齐相邻机制而把 QWEN_CI_VITEST_MAX_WORKERS 调成 25%：parseInt('25%') → 25，64 核宿主机上每个 Vitest 进程会跑 25 个 worker 而不是预期的 16 个，本要缓解的资源竞争随之复发——而两个钉住套件依旧全绿，因为它们钉的是表达式字面量，不是取值的语义。小数值同样解析错误：'0.5' → 0 个 worker。
证据：[探针] 在 64 核宿主机上驱动 vitest 3.2.7 的 resolveConfig：env=25% → poolOptions.threads.maxThreads=25、poolOptions.forks.maxForks=25（配置路径 maxWorkers:'25%' 解析为 16）；env=0.5 → maxThreads=0（tinypool 随后把 min 收缩到 max=0）；env=4 → 4。
修复建议：在工作流注释中写明取值必须是纯整数 worker 数，例如在现有注释后补充：'取值必须是纯整数 worker 数；Vitest 对这些环境变量执行 Number.parseInt，% 后缀会被截断为绝对数值。' 或者在导出前按整数模式校验该变量。修复约束：vitest 3.2.7 的解析点 maxThreads: Number.parseInt(process.env.VITEST_MAX_THREADS)（node_modules/vitest/dist/chunks/coverage.DfSpMS-b.js:3723）——'%' 分支对环境变量来源的取值不可达，修复不得经由此路径承诺百分比语义。
— qwen3.8-max via Qwen Code /review (v0.22.3)
wenshao
approved these changes
Sep 1, 2026
qwen-code-dev-bot
approved these changes
Sep 1, 2026
qwen-code-dev-bot
left a comment
qwen-code-dev-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed at head 84cf891f.
- Checked the three open review threads: all are bot-flagged
[Suggestion]maintainability notes (duplicated env block across ci.yml/release.yml, runner-name prefix as the ECS predicate,Number.parseIntsemantics if an operator ever sets a percentage string intovars.QWEN_CI_VITEST_MAX_WORKERS). None is a correctness defect: the variables are genuinely consumed by Vitest (verified against vitest 3.2.4's env parsing), the empty-string fallback keeps GitHub runners unchanged, and the current repo var is a plain number. They read as reasonable follow-up candidates, not blockers. - The pin tests match both copies, and the change is additive (+44/−0). No new Critical issues found.
- CI on this head is green.
This was referenced Sep 1, 2026
This was referenced Sep 1, 2026
This was referenced Sep 1, 2026
qwen-code-dev-bot
added a commit
to DragonnZhang/qwen-code
that referenced
this pull request
Sep 2, 2026
…10755) (QwenLM#10760) * fix(release): cap Vitest workers in the quality_scripts lane (QwenLM#10755) The 2026-09-02 nightly failed on the quality aggregate because the quality_scripts lane (added by the release-sharding split, QwenLM#10619) ran the scripts suite with vitest's default worker count — one worker per host core — on the shared ECS pool. On a 64-core runner executing several pool jobs at once, the suite's ESLint instances and bash replays spawned dozens of heavy subprocesses in parallel and 5-8 tests per run blew through the 30s test timeout non-deterministically (reproduced twice on the failing SHA: 30s timeouts across the ESLint boundary guards and workflow replays, plus a spawn-latency race in the upload-stall test). The sibling workspace_tests lane and the main CI gate already bound their vitest workers on ECS (QwenLM#10667); the new lane never inherited the bound. Apply the same tunable per-process bound to the lane and pin it in the release contract test next to the workspace lane's copy. Also deflake the upload-stall test: its 400ms kill bound raced child-spawn latency under pool load, losing attempt log lines and reading 1 or 2 attempts instead of 3; the bound is now 2s — still far under the shim's 60s hang, so the kill semantics are unchanged, and the worst-case run stays inside the test's own 30s budget. * refactor(release): alias the Vitest worker bound across release test lanes (QwenLM#10755) Review of the worker-cap fix noted that the Run Script Tests step carried a byte-identical copy of the Run Workspace Tests env block, so any future tuning (bumping the '4' default, adding another pool/thread variable) would need two synchronized edits, and editing one step while missing the other would silently restore the unbounded-worker timeout on the untouched lane — caught only reactively by a red contract test. Anchor the block at its first occurrence (&vitest_worker_bound) and alias it into the scripts lane, the same mechanism this file already uses for &release_test_env across six jobs. Reshape the contract assertions to pin the structure: the bound's four variables at the anchor definition and the bare alias at the consumer, so dropping either side turns the test red. Net five lines removed. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
This PR bounds each Vitest process on shared
ecs-qwen-*runners to one throughQWEN_CI_VITEST_MAX_WORKERSworkers for both thread and fork pools in the main CI unit-test job and the release quality job. The repository variable currently holds4; an unset variable falls back to4, and hosted runners keep their existing defaults.The bound is inherited by every parallel workspace Vitest invocation. It is a per-process limit rather than an aggregate four-worker limit for the entire runner, so workspace-level parallelism remains intact while the largest process-level multipliers are removed.
Why it's needed
The 128-core ECS hosts have many registered runners, and each runner sees the full host CPU count. The existing Core and CLI
25%setting can therefore resolve to about 32 workers per Vitest process, while workspaces without an explicit setting can approach the host CPU count. Because the root test command also runs workspaces in parallel, several concurrent runner jobs can multiply this into hundreds of competing workers.This is the failure shape in the main CI test job from run 33452224969: unrelated suites failed together, normally fast tests took 15–60 seconds, and failures appeared across Core, CLI, browser-bundle, and Web Shell tests while disk, inode, and memory samples remained healthy. A later main failure in run 33459716355 timed out an unchanged Web Shell boot test at 5,011 ms while its surrounding suite was 4.5–5.6 times slower than a green run.
The same pressure broke release run 33455887802: integration and audio-prebuild jobs passed, but workspace tests produced nine Vitest worker RPC timeouts followed by unrelated CLI, Core, and Web Shell failures, including 93 Web Shell timeouts.
Using a repository variable keeps the safe default in source while allowing maintainers to tune future runs without another workflow change. It does not attempt automatic host-load scheduling; that belongs in runner provisioning or cgroup/cpuset configuration.
Reviewer Test Plan
How to verify
Confirm that the main Ubuntu unit-test step and release
Run Workspace Testsstep set both Vitest max pool variables fromvars.QWEN_CI_VITEST_MAX_WORKERSwith a4fallback only onecs-qwen-*runners, keep both minimums at1, and leave hosted runners unchanged. Confirm that changing the repository variable affects newly started workflow jobs without modifying the workflow.Evidence (Before & After)
N/A — CI-only change. The two focused workflow suites passed 30 tests with 2 platform-specific skips. Prettier and ESLint passed for all changed files, and actionlint passed for the changed main CI workflow. Full/release actionlint remains blocked by pre-existing YAML alias diagnostics in the release workflow; the release workflow is parsed and its changed step is asserted by the focused regression suite.
Tested on
Environment (optional)
Node.js 22 with the focused Vitest workflow suites; local formatting, linting, and actionlint for the main CI workflow. Repository variable
QWEN_CI_VITEST_MAX_WORKERSis set to4.Risk & Scope
Linked Issues
Fixes #10665
Fixes #10669
Related to #10490, #10651, #10658, and #10663
中文说明
本 PR 做了什么
本 PR 为共享
ecs-qwen-*runner 上的每个 Vitest 进程设置最少 1、最多QWEN_CI_VITEST_MAX_WORKERS个 worker 的限制，同时覆盖 threads、forks、main CI 单元测试和 release 质量检查。仓库变量当前设置为4；变量未设置时默认使用4，GitHub hosted runner 继续使用原有默认值。该限制会被所有并行 workspace 的 Vitest 进程继承。它是每个进程的限制，不是整个 runner 总共只能运行 4 个 worker；workspace 级并行仍然保留，但最大的单进程并发放大器会受到控制。
为什么需要这个改动
128 核 ECS 上注册了多个 runner，每个 runner 都能看到整台宿主机的 CPU 数量。Core 和 CLI 现有的
25%配置因此可能让每个 Vitest 进程启动约 32 个 worker，没有显式配置的 workspace 甚至可能接近宿主机 CPU 数量。根测试命令还会并行执行多个 workspace，因此多个 runner job 同时运行时可能放大为数百个竞争 worker。main 的 run 33452224969 中，互不相关的 suite 同时失败，通常很快的测试耗时达到 15–60 秒，Core、CLI、browser bundle 和 Web Shell 都出现失败，而磁盘、inode 和内存保持健康。后续 run 33459716355 中，一个代码未变化的 Web Shell boot 测试在 5,011 毫秒超时，整个相邻 suite 比绿色运行慢约 4.5–5.6 倍。
同样的压力也导致 release run 33455887802 失败：integration 和 audio prebuild 均通过，但 workspace 测试出现 9 个 Vitest worker RPC timeout，并产生大量互不相关的 CLI、Core 和 Web Shell 失败，其中 Web Shell 有 93 个超时。
使用 repository variable 可以在源码中保留安全默认值，同时允许维护者无需修改 workflow 就调整后续运行。它不会根据宿主机负载自动调度；自动资源控制仍应由 runner provisioning 或 cgroup/cpuset 完成。
Reviewer Test Plan
如何验证
确认 main Ubuntu 单元测试 step 和 release 的
Run Workspace Testsstep 仅在ecs-qwen-*runner 上，从vars.QWEN_CI_VITEST_MAX_WORKERS读取 threads 和 forks 的最大值并在未设置时回退到4，两个最小值保持为1，hosted runner 保持原行为。确认修改仓库变量后，新启动的 workflow job 会读取新值而无需修改 workflow。证据（改动前后）
不适用——仅修改 CI。两个聚焦 workflow suite 通过 30 个测试，另有 2 个平台相关测试跳过。所有改动文件均通过 Prettier 和 ESLint，main CI workflow 通过 actionlint。全仓/release actionlint 仍被 release workflow 中既有的 YAML alias 诊断阻塞；聚焦回归测试已经解析 release workflow 并断言本次修改的 step。
测试平台
环境（可选）
Node.js 22，运行聚焦的 Vitest workflow suite，并在本地执行格式、lint 和 main CI workflow actionlint。仓库变量
QWEN_CI_VITEST_MAX_WORKERS已设置为4。风险与范围
关联问题
Fixes #10665
Fixes #10669
Related to #10490, #10651, #10658, and #10663