Harden tool-output budgeting, observability, and artifact lifecycle

[doudouOUC]

Status

Phase 1 correctness and its focused contract hardening are complete. The shared finalization implementation merged in #7323 on 2026-07-21, and the Shell no-artifact regression coverage plus persistedOutputFiles tri-state documentation merged in #7470 on 2026-07-22. #7323 also appeared in the automatically generated v0.20.1 changelog, so no manual edit to generated release files is needed.

Phase 2 is complete. The textual ACP and Headless transport bounds merged in #8450 and #9012, and #8447 is closed. Privacy-safe boundary diagnostics merged in #9039 on 2026-08-17, and #8448 is closed.

Phases 1 and 2 are complete. Phase 3 artifact lifecycle and Phase 4 budget evolution are deferred and should proceed only from focused, evidence-backed issues. Dynamic context-budget and masking proposals remain tracked in #2566 and #2567; generic frame limits, backpressure, replay aggregate limits, and daemon-wide resource protection remain tracked separately in #8091.

Final Phase 1 evidence:

• The focused finalizer reproduction went from 14,096 model-facing characters to exactly the configured 10,000-character budget while preserving both producer artifact references and writing no duplicate aggregate artifact.
• A live headless main-vs-PR comparison with a 5,000-character budget went from 60,980 characters on the wire on main to exactly 5,000 on the PR; the complete 80,276 characters remained recoverable in two producer artifacts.
• The final review covered 16 touched test suites with 1,692 passing, 1 skipped, and 0 failed tests; an independent real-disk finalizer harness passed 22/22 cases; the final PR CI was green.
• Interactive, headless, ACP, agent, and speculative paths now finalize at their model-submission boundary, and the recorded response matches the submitted response.
• The no-artifact producer-truncation path now has a direct Shell regression test: shortened content remains model-facing, the original oversized body is absent, and persistedOutputFiles is recorded as [] so the finalizer does not retry producer persistence.
• The code-side contract now documents all three persistedOutputFiles states: undefined means no producer persistence decision, [] means the producer decided but produced no reusable artifact, and a non-empty array contains reusable producer artifact paths.

Display-side baseline measurement

The 2026-07-23 synthetic measurement used character counts, UTF-8 byte counts, and SHA-256 hashes without logging output content. It found that MCP producer truncation changes llmContent but normally preserves the complete returnDisplay. ACP live maps the former into structured content and the latter into rawOutput before final model-response finalization, while headless JSON and stream-json also consume the complete display value.

• A single 499,999-byte ASCII result remained below the MCP per-part threshold. The finalized model-facing payload was 200,000 bytes, but the ACP SessionUpdate JSON was 1,000,281 bytes because the full result appeared in both content and rawOutput.
• Two 300,000-byte text parts each remained below the per-part threshold and aggregated to 600,001 bytes before finalization. The finalized model-facing payload was 200,000 bytes, but the ACP SessionUpdate JSON was 1,200,293 bytes.
• A 500,001-code-unit CJK result occupied 1,500,003 UTF-8 bytes. MCP reduced the producer model preview to 1,679 bytes, but the ACP SessionUpdate JSON remained 1,502,108 bytes because rawOutput retained the full display.
• Interactive retained history and recorded display metadata are already compacted to 32,000 JavaScript code units. They are not the primary amplification path, although character limits are not equivalent to UTF-8 byte limits.
• Headless JSON and stream-json currently emit the complete display value before finalization. Text mode does not print the tool body directly but may retain it in adapter memory.
• qwen serve already diagnoses pipe frames at or above 256 KiB using actual UTF-8 sizes, but it does not truncate, hash, or cover direct ACP and headless output.

The hashes confirmed that the large rawOutput/display payloads contained the original synthetic data rather than accounting overhead. The observed gap is therefore a real display/transport amplification issue, not a failure of the Phase 1 model-input budget.

Problem

Before Phase 1, tool outputs passed through several independent truncation and aggregation paths. Shell could truncate and persist an individual result before scheduling, other tools could be shortened by the scheduler, and a later batch-offload path applied a separate aggregate threshold. The batch path inferred state from rendered marker text and could skip results that were already producer-truncated. When several individually truncated Shell results were returned in one turn, their combined model-facing payload could therefore exceed the configured final budget.

The same conceptual batch could also be assembled in interactive, headless, ACP, agent, and speculative runtimes. Without a shared finalization boundary, the model request and the recorded transcript could diverge, persistence could be duplicated, and diagnostics could not reliably identify which layer changed a payload.

Root cause

There was no single authoritative final tool-response budget immediately before model submission. Persistence metadata was encoded in user-facing marker strings instead of structured state, and producer-level truncation was incorrectly treated as proof that the aggregate batch needed no further budgeting. Phase 1 addressed this root cause with a shared finalizer, structured persistence metadata, and a no-I/O send-boundary guard.

Display transport remains a separate concern: ACP and headless projections can serialize a complete display value after model-facing finalization, sometimes in two representations. Existing limits are measured in different units and applied at different boundaries, so they do not provide a stable wire-size contract.

Desired outcome

Every model request should receive a deterministic, bounded representation of tool responses. Full output should be persisted at most once when required, artifact references should remain available in the compact response, all runtimes should use the same finalization semantics, and the transcript should record exactly the parts submitted to the model.

Eligible textual ACP live/replay and headless tool-result fields should additionally have explicit UTF-8 JSON-byte budgets. The first Phase 2 implementation will keep both ACP content and rawOutput for compatibility and bound each independently; omitting or deduplicating equivalent wire fields is deferred until a compatible protocol decision is justified. Structured, rich, terminal, diff, media, and A2UI data must retain their current semantics unless a field is independently eligible for textual projection.

Delivery stages

• Phase 1 — Correctness and unified finalization: completed in fix(core): Enforce final tool response budgets #7323 and hardened in test(core): Cover Shell truncation without an artifact #7470. Introduced one shared finalizer; carried persisted artifacts as structured metadata; separated Shell and MCP persistence thresholds from their model preview budgets; covered interactive, headless, ACP, agent, and speculative runtimes; added a no-I/O send-boundary guard; ensured recorded parts equal submitted parts; documented and directly tested the producer-handled-without-artifact contract.
• Phase 2 — Display transport bounds and boundary diagnostics: completed under Bound textual tool-result display payloads across ACP and headless output #8447 and Add privacy-safe tool-result boundary diagnostics #8448. ACP PR 1 merged in fix(cli): Bound ACP textual tool-result payloads #8450, Headless PR 2 merged in fix(cli): Bound headless tool result content #9012, and privacy-safe diagnostics PR 3 merged in feat(core): Add privacy-safe tool-result boundary diagnostics #9039.
• Phase 3 — Artifact lifecycle: deferred. Unify tool-result and temporary-output artifact semantics only when a focused issue establishes concrete ownership, quota, cleanup, retention, or reuse requirements.
• Phase 4 — Budget evolution and rollout: deferred. Revisit byte- and token-aware budgets, structured/media policies, compatibility, metrics, and rollout only with current evidence; related proposals remain in feat(core): dynamic tool output truncation based on context pressure #2566 and feat(core): tool output masking service for context reclamation #2567.

Phase 2 final design

Fixed contract and scope

• Use a fixed 65,536-byte budget for each eligible textual projection. The limit is measured as the UTF-8 byte length of the field's JSON serialization, so JSON quotes, escapes, array wrappers, and content-block wrappers count toward the budget.
• Use an internal CLI-only projector. Do not add a public API, core export, user-facing configuration, capability negotiation, sidecar file, or new disk write.
• Keep ACP content and string rawOutput present in the first version and bound them independently. An identical content/raw string may reuse the same computed in-memory preview, but both fields remain serialized. Raw-field omission or wire deduplication is deferred.
• Keep structured _meta.artifacts unchanged in ACP PR 1. Artifact metadata budgeting and full ownership, retention, cleanup, and reuse semantics remain separate follow-up work and Phase 3 concerns.
• Generic NDJSON hard frame caps, backpressure, replay aggregate/container limits, and daemon-wide resource protection remain tracked by tracking(serve): Split daemon resource protection into reviewable PRs #8091. Phase 2 must not claim to solve those broader limits.

PR 1 — ACP textual tool-result projection

• Apply projection at both ACP delivery boundaries: live updates in Session.sendUpdate() and the history replay collector. Covering only Session.sendUpdate() would miss bulk replay/load paths.
• Do not project in the canonical transcript update constructor or replay state machine. Offline transcript export must remain lossless, and projection there would incorrectly make transport loss look canonical.
• Bound eligible pure-text content so JSON.stringify(content) is at most 65,536 UTF-8 bytes, including multi-block wrappers. Bound string rawOutput independently under the same rule.
• Use an O(n) scanner that accounts for native JSON escape expansion for ASCII, quotes, backslashes, controls, CJK, emoji, paired surrogates, and lone surrogates. Do not stringify the entire oversized value, repeatedly binary-search it, or join all blocks into another unbounded string.
• Produce a deterministic preview with approximately 20% head and 80% tail, including the truncation marker inside the budget. Force a bounded copy so an oversized backing string is not retained through a small slice.
• Share the content budget across text blocks with a deterministic water-fill allocation while preserving block count and order. If the empty-text structure alone exceeds the budget, collapse it to one bounded omission marker.
• Exempt the whole A2UI update using the existing A2UI metadata predicate because the bridge parses command JSON after the child wire boundary.
• Exempt structured rawOutput and mixed, diff, terminal, media, or otherwise non-pure-text content from that field's textual projection. An exemption for one field must not prevent projection of another independently eligible field.
• Preserve existing Shell/MCP artifact footers in the tail on a best-effort basis. Do not parse rendered truncation markers as state and do not claim that full output is recoverable when no producer artifact exists.
• Keep ordinary textual ACP fixture frames below 256 KiB after projection, but do not introduce or document a universal frame cap in this PR.
• Update the Phase 1 design statement that immediate ACP display events remain unchanged, because this PR intentionally changes their bounded transport projection.

PR 2 — Headless textual tool-result projection

• Apply projection after the adapter has selected the semantic tool-result value, immediately before BaseJsonOutputAdapter.emitToolResult() emits or retains it.
• Bound JSON.stringify(tool_result.content) to at most 65,536 UTF-8 bytes for eligible textual results.
• Cover JSON, stream-json, internal Text retention, SDK sessions, subagents, and DualOutput without gating on outputFormat.
• Do not inspect or inject persistedOutputFiles paths into Headless wire text. Existing producer footers may survive in the retained tail on a best-effort basis, while undefined, [], and non-empty metadata remain internal.
• Increment DUAL_OUTPUT_PROTOCOL_VERSION from 1 to 2 because downstream consumers observe a new bounded-content contract; keep the supported event set unchanged and update headless/DualOutput documentation. Do not change SDK schemas.
• Keep custom adapters, callers that emit prebuilt tool_result messages directly, overall JSON-array/session limits, stream backpressure, and upstream retention of a complete returnDisplay outside this PR.

PR 3 — Privacy-safe boundary length/hash diagnostics

• Enable diagnostics only when QWEN_DEBUG_LOG_FILE is configured and only for oversized or mutated representations. Do not publish content hashes as production metrics.
• Record internal stages for producer, finalizer input/output, recorder, ACP or headless projection input/output, and actual wire emission. Distinguish model text, display, ACP content/raw, headless content, and frame representations.
• Record JavaScript code-unit count, raw UTF-8 bytes, JSON UTF-8 bytes, whether the value changed, and artifact tri-state/kind.
• Use HMAC-SHA-256 with a process-random key. Hash each path or slot with length-delimited input instead of joining values. HMAC session, prompt, and tool-call identifiers as well; never log raw output, artifact paths, or identifiers. Values are intentionally comparable only within one process.
• Measure ACP bytes from a post-serialization write observation in the NDJSON stream and headless bytes at its writer boundary, so the reported wire size is the actual emitted size.
• Rate-limit diagnostic events to 50 per minute and report a suppressed-event count. Diagnostic failure must never change the tool result or transport path.
• Keep the existing large-pipe-frame observer as the qwen serve attribution mechanism for frames at or above 256 KiB; do not duplicate that responsibility.

Phase 2 acceptance criteria

• Threshold tests cover 65,535, 65,536, and 65,537 JSON bytes plus ASCII, CJK, emoji, quotes, backslashes, newlines, NUL/control characters, paired/lone surrogates, and seeded fuzz comparisons against native JSON serialization. Completed for ACP in fix(cli): Bound ACP textual tool-result payloads #8450.
• ACP regression fixtures cover one 499,999-byte ASCII result, two 300,000-byte text parts, one 500,001-code-unit CJK result, multi-block text, equal content/raw values, structured or mixed output, oversized A2UI, live delivery, resumed/bulk replay, and lossless offline transcript export. Completed in fix(cli): Bound ACP textual tool-result payloads #8450.
• Eligible ACP content and string rawOutput fields are each within 65,536 JSON UTF-8 bytes; both remain present, ordering is stable, payloads remain parseable, and ordinary textual fixture frames remain below 256 KiB. Completed in fix(cli): Bound ACP textual tool-result payloads #8450.
• Headless tests cover returnDisplay selection and fallback, error and vision results, all artifact tri-states, JSON and JSONL parseability, JSON/stream-json/Text/SDK/subagent/DualOutput paths, and preservation of model-facing responses and recordings. Completed in fix(cli): Bound headless tool result content #9012.
• Headless bounds apply to the target tool_result.content, not the entire accumulated JSON session or arbitrary custom message. Completed in fix(cli): Bound headless tool result content #9012.
• Diagnostic tests prove that unchanged representations retain the same HMAC within a process, the first mutating boundary produces the first mismatch, actual captured wire bytes match the recorded count, rate limiting reports suppression, and logs contain no raw content, paths, or identifiers. Completed in feat(core): Add privacy-safe tool-result boundary diagnostics #9039.
• Run focused Vitest suites, formatting and lint checks, build and typecheck verification, and a local fake-MCP end-to-end exercise. Completed for feat(core): Add privacy-safe tool-result boundary diagnostics #9039.
• Each implementation PR received the repository-required open-ended diff audits and clean final passes. Completed for feat(core): Add privacy-safe tool-result boundary diagnostics #9039.

Phase 1 acceptance criteria

• The combined model-facing tool-response payload never exceeds the configured final character budget, including batches containing multiple producer-truncated results.
• Producer persistence and aggregate fallback persistence do not write the same full output twice.
• Persisted artifact references survive final compaction.
• Small responses and non-tool user/model content remain unchanged.
• A persistence failure still yields a bounded model-facing response without fabricating an artifact reference.
• Interactive, headless, ACP, agent, and speculative paths share the same behavior.
• The response parts recorded in history are the same parts sent in the next model request.

Deferred follow-ups from Phase 1 review

These items were reviewed as non-blocking. Completed items are recorded here; the remaining items should be handled only as focused follow-ups when their impact justifies the work.

• Bound display-side payloads independently from model-facing previews while preserving canonical model responses and producer artifacts. ACP delivery completed in fix(cli): Bound ACP textual tool-result payloads #8450 and Headless delivery completed in fix(cli): Bound headless tool result content #9012.
• Document the persistedOutputFiles tri-state (undefined, [], and [paths]) in code. Completed in test(core): Cover Shell truncation without an artifact #7470.
• Remove redundant outer budget gates where the callee and shared finalizer already enforce the invariant.
• Avoid nested truncation headers when a finalized preview is reduced again.
• Revisit transcript crash durability between tool completion and final response recording if stronger durability requirements emerge.
• Add release-note visibility for Shell and MCP model-preview changes. fix(core): Enforce final tool response budgets #7323 is present in the automatically generated v0.20.1 changelog; generated release files were not edited manually.
• Add direct Shell regression coverage for producer truncation that changes content but produces no artifact. Completed in test(core): Cover Shell truncation without an artifact #7470.
• Define the status of suppressed, never-executed tool calls consistently across recording, replay, and analytics.
• Strengthen the ACP surrogate-pair preview regression across multiple tail-suffix alignments. Shared primitive tests added in fix(cli): Bound headless tool result content #9012 cover both -tail and -tail!!.

Next recommended step

Phase 2 is complete through #8450, #9012, and #9039. No further implementation is planned under this umbrella. Phase 3 and Phase 4 are deferred until concrete evidence justifies focused work; use #2566 and #2567 for context-budget evolution and #8091 for daemon-wide resource protection.

Out of scope for Phase 1

Wire-level byte/hash telemetry, exact tokenizer-based accounting, media budgeting, display transport bounds, artifact cleanup policy, and a broader temporary-file redesign remain intentionally deferred to the later stages above.

[qwen-code-ci-bot]

Triage (re-run 2026-08-20)

• Type: enhancement — umbrella RFC, closed as completed by the author today
• Labels: type/enhancement, category/core, scope/core, priority/P2 — removed stale need-discussion (discussion happened; work delivered)
• Next: none — closure verified; deferred work tracked in feat(core): dynamic tool output truncation based on context pressure #2566, feat(core): tool output masking service for context reclamation #2567, tracking(serve): Split daemon resource protection into reviewable PRs #8091

Verified against the repository (all closing-comment claims hold):

• Phase 1 merged: fix(core): Enforce final tool response budgets #7323 (2026-07-21), test(core): Cover Shell truncation without an artifact #7470 (2026-07-22)
• Phase 2 merged: fix(cli): Bound ACP textual tool-result payloads #8450 (2026-08-05), fix(cli): Bound headless tool result content #9012 (2026-08-12), feat(core): Add privacy-safe tool-result boundary diagnostics #9039 (2026-08-17); sub-issues Bound textual tool-result display payloads across ACP and headless output #8447 and Add privacy-safe tool-result boundary diagnostics #8448 closed
• Deferred tracking issues all open: feat(core): dynamic tool output truncation based on context pressure #2566 (dynamic context budget), feat(core): tool output masking service for context reclamation #2567 (output masking), tracking(serve): Split daemon resource protection into reviewable PRs #8091 (daemon resource protection)

Stage 2 — product-direction verdict

Closure accepted. The Phase 1–2 scope was aligned with product direction (correctness of the core model-input budget, deterministic wire bounds, shared finalization across runtimes) and shipped through individually reviewed, maintainer-merged PRs. The original triage recommendation — split the speculative phases into focused issues — was followed: Phase 3/4 were deferred to #2566, #2567, and #8091 instead of keeping this umbrella open indefinitely. No reopen needed; future work should be filed as narrowly scoped issues.

中文说明

• 类型: enhancement —— 总览 RFC，作者今日已按完成关闭
• 标签: type/enhancement, category/core, scope/core, priority/P2 —— 已移除过期的 need-discussion（讨论已完成、工作已交付）
• 下一步: 无 —— 关闭已核实；延期工作由 feat(core): dynamic tool output truncation based on context pressure #2566、feat(core): tool output masking service for context reclamation #2567、tracking(serve): Split daemon resource protection into reviewable PRs #8091 跟踪

已对照仓库核实（关闭评论中的声明全部成立）：

• Phase 1 已合并：fix(core): Enforce final tool response budgets #7323（2026-07-21）、test(core): Cover Shell truncation without an artifact #7470（2026-07-22）
• Phase 2 已合并：fix(cli): Bound ACP textual tool-result payloads #8450（2026-08-05）、fix(cli): Bound headless tool result content #9012（2026-08-12）、feat(core): Add privacy-safe tool-result boundary diagnostics #9039（2026-08-17）；子 issue Bound textual tool-result display payloads across ACP and headless output #8447、Add privacy-safe tool-result boundary diagnostics #8448 已关闭
• 延期跟踪 issue 均为 open：feat(core): dynamic tool output truncation based on context pressure #2566（动态上下文预算）、feat(core): tool output masking service for context reclamation #2567（输出掩码）、tracking(serve): Split daemon resource protection into reviewable PRs #8091（守护进程资源保护）

Stage 2 产品方向结论：认可关闭。Phase 1–2 范围与产品方向一致（核心模型输入预算的正确性、确定性的传输边界、跨运行时共享的 finalization），并通过逐个评审、维护者合并的 PR 交付。初次 triage 的建议——将推测性阶段拆分为聚焦 issue——已被遵循：Phase 3/4 延期至 #2566、#2567、#8091，而非让本总览 issue 无限期挂起。无需重开；后续工作应以窄范围 issue 提出。

--- Qwen Code

[doudouOUC]

Phase 1 implementation is available in #7323. The PR uses Refs #7306 so this umbrella issue remains open for the wire diagnostics, artifact lifecycle, and adaptive budgeting follow-up phases.