Optimize daemon cold start and qwen serve fast-path latency

[doudouOUC]

Background

The original benchmark on the early daemon implementation showed a large cold-start gap: daemon boot plus the first session was about 2.5s, while a full CLI initialization was about 0.7s. The listener/health path has since been substantially optimized, so this issue now tracks the remaining post-health cold first-session latency rather than treating the original 2.5s breakdown as current.

Current status (updated after the P1-B measurement gate, 2026-07-19)

Area	Current state
Process start → first /health	The health-first architecture remains in place. On the current 2C4G representative host, the exact #6907 merge commit reached the listener in 641.1ms P50 and completed /health in another 6.7ms P50. Historical numbers from other hosts are not directly comparable.
Runtime and ACP startup	Runtime starts after the first health response (or a fallback timer), then daemon ACP preheat runs asynchronously. #7200 now advertises independent v1 status and preheat capabilities. A capable Web UI uses them only for the exact primary workspace and starts best-effort preheat in its deferred no-session flow, but it does not await preheat before the first Session; that request can still join an in-flight channel initialization.
Cold first Session	The current 30-run baseline is 1251.5ms P50 / 1302.2ms P95 for POST /session, or 1911.3ms P50 / 2016.7ms P95 from process start to Session success.
Dominant stage	The #7145 P0-A profile attributes 67.3% of ACP child startup P50 to Gemini/ACP module loading. The #7182 matched 2C4G comparison reduced ACP import P50 from 115.06ms to 52.00ms and channel.initialize P50 from 1098.25ms to 1035.61ms.
Session creation	session.new is 62.0ms P50 / 65.9ms P95. The ACP child session-start profile is 57.5ms P50, dominated internally by config_setup at 51.0ms P50. This is not the leading cold-path bottleneck.
Warm/already-preheated Session	The second Session in the same process is 77.8ms P50 / 101.4ms P95. A first Session sent after a 3s preheat wait is 76.0ms P50 across three validation runs.
MCP reuse	A workspace-scoped McpTransportPool is present on current main, so the old cross-session MCP-pool item is no longer unimplemented. Current ACP Session creation does not block success on waitForMcpReady(); MCP failures are surfaced asynchronously.
Profiling	#6349 profiles stages inside GeminiClient.startChat; #6907 provides runtime/channel/session/child correlation; #7145 adds bounded channel.initialize phase attribution; #7182 applies and validates the selected narrow module-loading optimization.
ACP readiness contract	#7200 completed P1-A. P1-B then tested earlier explicit preheat and a bounded wait on the same release bundle; neither produced a qualifying user-visible win, so the current fail-open, Session-authoritative behavior remains unchanged.

Current representative baseline

The benchmark used the exact #6907 merge commit 10a39d15d278c745a822293846ce3297e6bd1a6c on an Alibaba Cloud Linux 4.0.3 ECS with 2 vCPU, approximately 3.48GiB RAM, no swap, and the bundled Node.js v22.23.1 runtime. It ran one excluded warm-up followed by 30 serialized cold daemon/ACP process starts, a second warm Session in every process, three already-preheated samples, one two-request cold concurrency sample, one telemetry-disabled sample, and one default-single REST compatibility sample.

The formal performance requests used sessionScope: thread so every cold, warm, and concurrent request had a distinct Session ID for trace/profile correlation. This does not alter deferred runtime or ACP channel startup. A separate request omitted sessionScope and verified that the legacy default-single response still succeeds with the original response fields.

End-to-end and resource results

Metric	Samples	P50	P95	Max
Process start → listener	30	641.1ms	749.4ms	788.4ms
/health request	30	6.7ms	16.8ms	17.6ms
Cold POST /session	30	1251.5ms	1302.2ms	1305.3ms
Process start → first Session success	30	1911.3ms	2016.7ms	2035.6ms
Second warm Session	30	77.8ms	101.4ms	111.7ms
First Session after a 3s preheat wait	3	76.0ms	76.6ms	76.6ms
Process-tree RSS at listener	30	174.2MiB	188.3MiB	188.6MiB
Process-tree RSS after first Session	30	415.9MiB	440.5MiB	442.3MiB
50ms-sampled process-tree peak RSS	30	422.1MiB	440.5MiB	442.3MiB

Cold Session request decomposition

Stage	P50	P95	Result
Deferred runtime wait	173.0ms	192.2ms	30/30 runtime.path=started_on_request
channel.wait	997.1ms	1040.4ms	30/30 channel.path=joined
session.new	62.0ms	65.9ms	ACP Session creation after channel readiness
Other request/middleware time	20.4ms	21.9ms	Request span minus the three non-overlapping stages

channel.wait represents 79.49% of the request span at P50 and has a 0.916 correlation with client-observed cold Session latency. All 30 Session traces link by channel UUID to the automatic preheat trace: channel.spawn is 12.4ms P50 / 13.3ms P95, while channel.initialize is 1006.5ms P50 / 1050.1ms P95. The next performance investigation should therefore split channel.initialize; optimizing process spawn or session.new first would target much smaller stages.

Functional and safety results

• Cold Session success: 30/30; already-preheated success: 3/3; all formal cold processes exited with code 0.
• Two simultaneous cold first-Session requests both succeeded with distinct Session IDs, shared one channel UUID, and joined the same in-flight preheat.
• The default single request, with sessionScope omitted, returned HTTP 200 and the original sessionId, workspaceCwd, attached, clientId, and createdAt response shape.
• Session creation succeeded with telemetry and the detailed profiler disabled, without producing telemetry/profile files.
• All 30 cold runtime/channel/session/child trace chains and all 60 cold/warm Session-ID profile correlations were complete. All 720 top-level cold telemetry records parsed successfully.
• No OOM, kernel kill, benchmark daemon, ACP child, or listening port remained after the run.

P0 completion result (#7145 + #7182)

P0-A #7145 added an opt-in, bounded ACP initialize profile without changing the readiness barrier, timeout/cleanup behavior, failure semantics, or Session contract. On the representative 2C4G host, the profile attributed 67.3% of child startup P50 to Gemini and ACP module loading.

P0-B #7182 selected the narrow module-loading optimization: pure classifier/suggestion contracts no longer import through TUI modules, and low-frequency interactive dependencies are loaded only when their actions execute. The emitted-bundle guard now rejects Ink, React, React Reconciler, or Yoga in the ACP static closure while allowing intentional dynamic imports.

The release-built comparison used 60 alternating cold control/candidate pairs after excluded warmups, plus 30 alternating preheated pairs. No samples were discarded.

Metric	Matched control	#7182 candidate	Change
ACP import	115.06ms P50	52.00ms P50	-63.06ms (-54.8%)
Child process → initialize response	1102.88ms P50	1041.09ms P50	-61.80ms
channel.initialize	1098.25ms P50	1035.61ms P50	-62.64ms
Process → first Session	2046.88ms P50	1980.03ms P50	-66.85ms
Cold Session request	1358.95ms P50	1290.23ms P50	-68.72ms
Warm Session	137.53ms P95	104.98ms P95	-32.55ms
First health	962.99ms P95	824.14ms P95	-138.85ms
Process-tree RSS	442.27MiB P95	435.70MiB P95	-6.57MiB

All 60 cold profiles and all 30 preheated profiles were complete. Concurrent cold first Sessions, telemetry-disabled startup, legacy default-single behavior, release-bundled ACP initialize/newSession, clean shutdown, and residual-process checks passed. The ACP static closure decreased from 202 chunks / 18.75MiB to 198 chunks / 17.59MiB, with TUI-bearing chunks reduced from 2 to 0. #7182 merged as 86ad532deba05257c241095984566ac49f537a0a on 2026-07-19.

The benchmark intentionally did not drop the Linux page cache because that would disturb other workloads and does not represent repeated process starts on a normal host. A pre-existing unrelated Qwen process tree was left running; it was essentially CPU-idle and was excluded from process-tree RSS measurements.

Historical downstream evidence from 0.19.3-preview.2

A representative downstream deployment previously collected 10 cold starts with 2534ms P50 from /health success to Session success, including a 445ms capabilities request and a 1713ms Session POST. The delay between health and sending the Session request was negatively correlated with Session POST duration (approximately -0.71), consistent with waiting for the remainder of an in-flight ACP preheat. These values remain historical context only; the current 30-run baseline above is the acceptance reference for the #6907 merge commit on the specified 2C4G environment.

P1-A completion result (#7200)

#7200 completed the contract and compatibility gate without moving the Session barrier. The daemon advertises workspace_acp_status and workspace_acp_preheat independently; both remain singular, primary-workspace REST control-plane routes. Status is a point-in-time snapshot, preheat preserves ready === channelLive, and each caller has its own timeout while the service retains one shared in-flight initialization until settlement. Timeout or failure remains best effort, sanitized to clients, detailed in daemon logs, and retryable through the existing Session path.

The TypeScript SDK now routes both calls through REST regardless of the configured Session transport. The Web UI requires the advertised capability and an exact primary-workspace match; old daemons and secondary workspaces retain the existing lazy Session path. The Web UI continues to start best-effort preheat after its deferred provider, skill, ACP-status, and Git bootstrap batch settles, and it does not await readiness before first Session creation. P1-B measured alternatives to that timing and rejected them because they did not improve total user-visible latency.

#7200 merged as 6872b48c28fbbe6b190daaa778f0e4ca96473a73 on 2026-07-19. The PR was approved and all 10 required checks passed. It intentionally makes no latency-improvement claim.

P1-B completion result (no production patch)

P1-B used a release bundle from common source commit 0d03bf7f0f5461b9d11c5ce4f5a85a0ee5d87274 for both control and candidate orchestration. #7200 remains the protocol reference, not the causal performance control. On the representative Alibaba Cloud Linux 2C4G host, the formal run excluded one warmup, then executed 30 alternating current/early pairs at each 0ms, 250ms, 500ms, and 1000ms connected-to-prompt delay, plus 10 direct-after-bootstrap and 10 await diagnostics. All 261 processes succeeded, returned explicit preheat ready: true, exited with code zero, and left no residual process.

Connected-to-prompt delay	Paired action → Session P50, early minus current	Paired process → Session P50, early minus current	Decision
0ms	-2.75ms	-2.80ms	Below gate
250ms	-1.21ms	+2.29ms	Below gate / process regression
500ms	-7.44ms	-9.01ms	Largest improvement, still below 10ms gate
1000ms	+0.05ms	-3.82ms	No action-time improvement

No cohort met the required 10ms P50 user-visible improvement. Tail movement was inconsistent, including a 33.65ms process-to-Session P95 regression at 0ms and a 31.74ms action-to-Session P95 regression at 250ms. RSS P95 exceeded the 5MiB non-regression gate in three cohorts; because both sides used the same bundle and the candidate adds no production allocation, this is treated as tail noise but still cannot support accepting a zero-benefit change.

The await diagnostic reduced Session POST P50 from 927.09ms to 47.30ms only by adding 877.50ms P50 immediately before the POST. Total action-to-Session remained 927.11ms versus 927.25ms and process-to-Session remained 1795.27ms versus 1797.78ms. This confirms that awaiting preheat changes attribution rather than readiness time. Removing the ACP status request without moving preheat also showed no total-time benefit.

The code path explains the result: production qwen serve already starts automatic bridge preheat before runtime capabilities are available, and explicit preheat and the first Session join the same in-flight channel initialization. Starting an explicit request earlier cannot advance that work; awaiting it adds an HTTP response/request handoff and may join an automatic attempt that later fails instead of issuing a later retry. P1-B is therefore evidence-complete with no timing, promise, status, or Session-barrier production change. P1-A and the authoritative fail-open Session retry path remain unchanged.

Current cold first-session path

process start
  → bootstrap listener
  → first /health response
  → deferred runtime load
  → bridge.preheat() [fire-and-forget]
      → ACP child spawn
      → ACP child initialize
  → first Session request
      → join in-flight / reuse ready channel
      → ACP session/new
          → settings + Config construction
          → config.initialize() [MCP discovery progresses asynchronously]
          → authentication
          → GeminiClient.startChat()

Completed work

Work	Status
Skip unnecessary ACP relaunch, daemon-time preheat, and idle keep-alive	#4751; present on current main
Workspace-scoped MCP transport sharing across sessions	#4336; present on current main
Serve fast-path and deferred runtime mount	#5785
Skip the extra spawnSync wrapper for qwen serve	#5874
Start CLI version lookup asynchronously and enable the Node compile cache	#5938
Remove serve bridge re-export shims and break the eager ACP runtime import path	#5955 and #5989
Route standalone qwen serve through the fast-path shim	#5977
Add bundle-level fast-path regression coverage	#5995
Flush first /health before loading the runtime	#6013
Add GeminiClient.startChat stage profiling	#6349
Reduce Git snapshot process overhead in startChat	#6784
Add correlated cold first-session tracing across daemon runtime admission, ACP channel startup/reuse, child session/new, and the detailed startChat profile	#6907; merged 2026-07-16
Re-baseline the merged implementation with at least 30 cold starts, warm/preheated/concurrent/legacy compatibility coverage, tail latency, and process-tree RSS on a representative 2C4G host	Completed 2026-07-17; results recorded above
Add bounded channel.initialize phase profiling without changing startup or Session semantics	#7145; merged 2026-07-18
Remove TUI-only modules from the ACP static startup closure and add a bundle-level regression guard	#7182; merged 2026-07-19; P0 results recorded above
Define and advertise the primary-workspace ACP status/preheat contract, preserve shared preheat across waiter timeouts, and gate SDK/Web UI compatibility	#7200; merged 2026-07-19; P1-A complete
Evaluate earlier explicit preheat, status removal, and an awaited readiness handoff on the representative 2C4G path	Completed 2026-07-19; P1-B rejected by the measurement gate, so no production patch

Remaining work

P0, P1-A, and P1-B are complete. There is no remaining readiness-contract or first-Session timing change supported by the current evidence. The item below is optional follow-up work with its own profiling and acceptance gate; it is not required to complete this issue.

P2 — Optimize warm Session setup only after the cold bottleneck

If a separate warm-Session target is required, split the approximately 35ms P50 portion of config_setup outside startChat(). This is meaningful within the 76–78ms warm path but only about 2.8% of the current cold end-to-end request, so it should not displace the channel.initialize work.

Small listener fast-path follow-ups such as deferring the final CLI version await, replacing the bootstrap Express app, or parallelizing fast-path settings reads should also remain measurement-driven; they do not explain the dominant post-health delay in the current baseline.

Superseded conclusions from the original issue

• P0 is complete, not pending: feat(daemon): Profile ACP channel initialization #7145 provides bounded channel.initialize attribution and perf(cli): Defer TUI runtime from ACP startup #7182 applies the selected narrow module-loading optimization with release-built 2C4G acceptance evidence.
• The current-release re-baseline is complete, not pending: the 30-run feat(daemon): Trace cold first-session startup #6907 benchmark and functional/resource checks are recorded above.
• P0-7 is complete, not pending: fix(cli): Avoid ACP runtime preload on serve fast path #5989 imports the bridge type and workspace-path constant through lightweight subpaths, and fix(cli): Guard serve fast-path bundle closure #5995 guards the emitted bundle closure.
• Cross-session MCP pooling is complete, not pending: feat(serve): shared MCP transport pool [F2] #4336 added the workspace-scoped transport pool now used by Session configs.
• waitForMcpReady() is not on the current ACP Session success critical path. MCP discovery can complete progressively after Session creation.
• ACP preheat is not guaranteed to finish before the first Session. After the health-first startup change it is intentionally fire-and-forget, so a cold Session may wait for it.
• session.new is not the current leading cold bottleneck. The current evidence attributes approximately 1s to channel.initialize before the approximately 62ms Session request.
• The original warm-session, memory, and stage-breakdown numbers were measured on an older implementation and should not be presented as current without the new baseline above.

Completion criteria

• P0 complete: channel.initialize sub-stages are measured on the representative path, and perf(cli): Defer TUI runtime from ACP startup #7182 reduces the selected phase and total process → Session time without regressing first-health latency, warm Session latency, failure fallback, or process-tree RSS.
• P1-A complete: feat(daemon): Advertise ACP preheat readiness #7200 advertises the primary-workspace status/preheat capabilities, defines point-in-time readiness and failure semantics, preserves service-level single-flight across waiter timeouts, routes SDK calls through REST, and keeps old-daemon, secondary-workspace, and lazy-Session compatibility.
• P1-B complete: same-source release-built testing demonstrated that earlier explicit preheat does not meet the 10ms user-visible P50 gate and that awaiting readiness only relocates latency. No production timing or Session-barrier change is warranted.
• Regression evidence remains complete for cold, in-flight-preheat, already-preheated, concurrent, telemetry-disabled, legacy default-single, release-bundled ACP, and clean-shutdown paths. The P1-B candidate-only compatibility matrix was not expanded after the performance gate rejected the candidate before any production code existed.
• The issue-level cold-start and readiness goals are complete. Optional P2 warm-Session config_setup work must be tracked and accepted independently; it is not a continuation of the readiness contract.

Key files

• packages/cli/src/serve/run-qwen-serve.ts — health-first startup, deferred runtime load, automatic preheat
• packages/cli/src/serve/capabilities.ts — advertised ACP status/preheat feature tags
• packages/cli/src/serve/routes/workspace-status.ts — explicit ACP preheat/status routes
• packages/cli/src/serve/workspace-service/index.ts — readiness, per-waiter timeout, shared preheat, and retry semantics
• packages/acp-bridge/src/bridge.ts — channel spawn/reuse and preheat
• packages/sdk-typescript/src/daemon/DaemonClient.ts — REST-routed ACP status/preheat client calls
• packages/webui/src/daemon/session/DaemonSessionProvider.tsx — capability/workspace gating and deferred best-effort preheat
• docs/design/acp-preheat-contract.md — P1-A contract, compatibility, and non-goals
• packages/cli/src/acp-integration/acpAgent.ts — per-session config, auth, and Session creation
• packages/core/src/core/client.ts — GeminiClient.startChat profiling
• packages/core/src/telemetry/daemon-tracing.ts — daemon/ACP trace propagation and correlated cold-session spans
• packages/core/src/core/session-start-profiler.ts — Session-ID-correlated JSONL profiling
• packages/cli/src/utils/acp-startup-profiler.ts — fixed, bounded ACP child startup phase collection
• packages/acp-bridge/src/channel-startup-profile.ts — parent-side startup profile validation and span attributes
• scripts/check-serve-fast-path-bundle.js — serve and ACP static import-closure regression guards
• integration-tests/cli/qwen-daemon-startup-benchmark.test.ts — daemon startup benchmark
• integration-tests/cli/qwen-daemon-vs-cli-benchmark.test.ts — daemon vs CLI benchmark

🤖 Generated with Qwen Code

[qwen-code-ci-bot]

This appears related to the broader daemon/serve roadmap:

• Daemon mode (qwen serve): proposal & open decisions #3803 - Daemon mode (qwen serve): proposal & open decisions
• proposal(serve): qwen --serve (Mode A) — TUI + in-process HTTP daemon, 3-phase plan (Stage 1.5b) #4156 - proposal(serve): qwen --serve (Mode A) — TUI + in-process HTTP daemon, 3-phase plan
• Background task management: roadmap and next steps #3634 - Background task management: roadmap and next steps
• proposal(serve): L2 能力分层 — 抽出 DaemonWorkspaceService，收口 file/auth/agents/memory（/acp↔REST 等价的前置） #4542 - proposal(serve): L2 能力分层 — 抽出 DaemonWorkspaceService

These issues contain foundational context for the daemon architecture and related performance considerations.

[doudouOUC]

Update: CLI wrapper fast-path for qwen serve (A1 optimization)

PR #5874 (merged) adds a new optimization not previously tracked in this issue:

Skip spawnSync in cli-entry.js for the serve path

The qwen bin wrapper previously used spawnSync(node, ['--expose-gc', 'cli.js', ...args]) for all commands. Since the daemon host process never calls global.gc() (only ACP children do, via their own --expose-gc in spawnChannel.ts), the serve command now imports cli.js directly in-process.

Result: Eliminates one full Node.js process startup from the daemon boot path (~370ms on EDR-instrumented hosts, ~40% reduction in wall-clock cold start time).

Benchmark (macOS/arm64, EDR environment):

	Before	After
Wall clock to /health 200	~1200ms	~345ms

This is additive to the P0 optimizations already completed (skip relaunch, preheat, keep-alive).