feat(vis): full session debugging — tasks/cron, execution timeline, retries & tool progress#1210
Conversation
The visualizer read every wire/state/blob artifact a session persists but ignored the two on-demand families agent-core also writes under the session directory: background tasks (tasks/<id>.json + output.log) and cron jobs (cron/<id>.json). Neither is reconstructable from the wire, so there was no way to inspect what a session spawned in the background or scheduled. Server: - task-store / cron-store read-only readers mirroring agent-core's on-disk layout, id-validation guard, and legacy snake_case task normalization - GET /:id/tasks, /:id/tasks/:taskId/output (byte-window paged via an exact nextOffset cursor), and /:id/cron routes - re-export the public background-task types from agent-core; mirror the non-exported CronTask shape with a fixture-backed drift test Web: - Tasks tab: process/agent/question kinds with status, timing, kind-specific fields, raw JSON, and a progressively paged output.log viewer - Cron tab: expression, prompt, recurring/one-shot, created/last-fired - count badges on both tabs Tests: +20 (lib + route), all 113 vis-server tests green; web typecheck and build clean.
Two transient signals were only ever emitted as live-only loop events, so nothing survived in the agent record for post-hoc analysis: - step retries: chatWithRetry gains an onRetry callback; turn-step collects the recovered attempts and attaches them to step.end as an optional `retries` array (previously only the live `step.retrying` event). - tool progress: tool-call distills a tool's sparse status/percent updates into a bounded `progress` summary (updateCount / lastStatus / maxPercent) on tool.result. Streamed stdout/stderr is excluded — it would bloat the wire and is already reflected in the result output. Both are additive optional fields, so the wire protocol version is unchanged and existing records keep loading. New public types: LoopStepRetryRecord, LoopToolProgressSummary.
Turn the debugger from a flat record viewer into an analysis tool. New Timeline tab: folds the wire into turns → steps → tool calls (client-side, no extra round-trip) and derives the metrics the raw list hides — per-turn / per-step / per-tool duration, per-turn token cost, a context-window fill sparkline with cache-hit rate, a tool usage table, idle-gap detection, and a config-change timeline. Inline elsewhere: - Wire rows show tool.call → tool.result elapsed time; tool.result detail shows truncation, output size, retries, and the progress summary. - Issues drawer gains tool-error, truncation, filtered, max_tokens, and retried categories. - Tasks tab links agent-kind tasks to the subagent's wire. Wires up vitest for the web package and adds analysis/issues unit tests.
|
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ff2d7185ee
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ltering A `/export-debug-zip` bundle is just `manifest.json` plus a flattened session directory, which vis already knows how to read. Importing one therefore lights up every existing tab for a session that lives on someone else's machine. Server: - zip-import: yauzl extraction with zip-slip path guards and entry-count / uncompressed-size caps for untrusted uploads. - import-store: extract a bundle into <home>/imported/<imp_…>/, validate it has a main wire, and record an import-meta.json sidecar. - session-store resolves imp_-prefixed ids against imported/, so wire / context / tasks / cron / blobs / logs all work on imported sessions; agent homedirs are re-derived locally (the bundle holds foreign absolute paths). - POST /api/imports (raw zip body) and GET /api/sessions/:id/logs (structured log lines — also available for local sessions). Web: - session rail: import button + all/local/imported filter + imported badge. - new Logs tab: virtualized, level filter, search, session/global toggle. - manifest card atop the State tab for imported sessions. SessionSummary/SessionDetail gain `imported` + `importMeta`. Tests cover extraction, the zip-slip guard, list merge, reading an imported wire through the existing route, and log parsing.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2bb23c4b1f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…l status text Addresses review feedback on the debug-tooling changes: - Background tasks and cron jobs are persisted under each agent's homedir (<session>/agents/<id>/tasks and /cron), not the session root. The Tasks and Cron tabs read the session root, so they showed nothing for normal sessions. Both routes now aggregate across detail.agents homedirs; task entries carry the owning agentId. The route-test fixtures were writing to the wrong (session-root) location too — corrected to the real agents/main layout so they actually exercise the path. - tool.result progress no longer keeps free-form status text, only updateCount and maxPercent. A tool's status string can contain sensitive data (e.g. an MCP OAuth authorization URL) that must not leak into persisted wire files or exported debug bundles.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c3d558a3a8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The <main> flex child lacked min-w-0, so it defaulted to min-width:auto and refused to shrink below its content's intrinsic width. Tabs that lay out in normal flow with flex-wrap rows (the Timeline tab) then got unbounded width, never wrapped, and blew the layout out to thousands of pixels wide. Adding min-w-0 lets the column shrink to the available width so its content wraps, truncates, or scrolls within its own container.
# Conflicts: # packages/agent-core/src/loop/tool-call.ts
…back - Logs tab: for non-imported sessions the shared global log lives at <KIMI_CODE_HOME>/logs/kimi-code.log, not under the session dir (that path is only used inside exported bundles). The route now reads the home path for local sessions, so the global-log toggle works for them. - Imported detail: a bundle's state.json is best-effort and may omit the agents map. When the inventory is empty, fall back to discovering agents from disk so routes that require an agent (wire/context) still resolve main.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 162708e90e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…nput
An imported debug zip is untrusted, so a syntactically valid but type-corrupt
file could crash whole views:
- manifest.json: a non-string field (e.g. workspaceDir: 123) flowed into
SessionSummary.workDir, where the session rail calls .split('/') and crashed
the entire list. readManifest/readImportMeta now sanitize declared string
fields, keeping only strings.
- task JSON: a record that passed the shape guard but held a non-string legacy
field (e.g. stop_reason: 5) threw in normalization, failing GET /tasks with a
500 and hiding all of a session's tasks. optionalNonEmptyString now tolerates
non-strings, and listBackgroundTasks skips any record that still fails to
normalize — honouring the reader's documented silently-skips contract.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9a56d7b90a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| // A debug bundle must contain a main wire; without it there is nothing to | ||
| // visualize. `state.json` / `manifest.json` are best-effort. | ||
| const hasMainWire = await pathExists(join(dir, 'agents', 'main', 'wire.jsonl')); |
There was a problem hiding this comment.
When an imported /export-debug-zip comes from a session that stores the main transcript as root wire.jsonl rather than agents/main/wire.jsonl (the export path still includes arbitrary session files, and packages/node-sdk/test/export-session.test.ts covers root-wire bundles), this validation rejects the upload as not a session bundle. That prevents users from inspecting valid older/debug exports; accept root wire.jsonl and synthesize it as the main agent when the agent-scoped wire is absent.
Useful? React with 👍 / 👎.
- Logs tab: the diagnostic log can rotate (kimi-code.log.1, .2, …) and an exported bundle may contain only the archives. The route now discovers the active file plus its rotated siblings and concatenates them oldest-first, so a rotated-away log still surfaces (covered by node-sdk's rotated-export case). - agent-core: a tool that reported sparse progress and then threw lost its progress summary, because the catch path built the error tool.result without it. Thread progressSummary through that path too, matching the success and malformed-return paths.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6d6c37d58e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
readImportedDetail's empty-inventory fallback never ran when a bundle's
state.json had a non-empty but type-corrupt agents map (e.g.
`{ "agents": { "main": null } }`): inventoryAgents dereferenced the null entry
and threw, so readSessionDetail returned 500 instead of recovering main from
the on-disk agents/main/wire.jsonl. inventoryAgents now skips non-object
entries, letting the disk-discovery fallback take over.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a4b5d1fa9a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… zero-usage steps - Timeline tab kept the previously-selected agent id across session navigation, so a subagent selection would 404 against the next session. Reset it to main on sessionId change, mirroring WireTab/ContextTab. - A zero-usage step.end (e.g. a content-filtered response) reset the context-window fill to 0, pushing a false drop into the Timeline chart and the Context tab. agent-core's ContextMemory keeps the prior count in that case; the analysis lib and the context projector now do the same.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d29b02f915
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const name = c.req.query('name') ?? null; | ||
| let buffer: Buffer; | ||
| try { | ||
| buffer = Buffer.from(await c.req.arrayBuffer()); |
There was a problem hiding this comment.
When a client uploads with no Content-Length (for example chunked transfer) or a falsely small length, this buffers the entire request before the MAX_ZIP_BYTES check runs. A bad /api/imports request can therefore allocate an arbitrarily large ArrayBuffer and hang or OOM the vis server even though the route intends to cap uploads; stream the body and abort once the byte limit is crossed.
Useful? React with 👍 / 👎.
| const buffer = Buffer.allocUnsafe(length); | ||
| const { bytesRead } = await handle.read(buffer, 0, length, start); | ||
| const content = buffer.toString('utf-8', 0, bytesRead); |
There was a problem hiding this comment.
When a large output.log page boundary falls inside a multi-byte UTF-8 character, decoding each byte window independently replaces the split bytes with �; the next page starts after the remaining bytes, so the concatenated log shown in the Tasks tab is permanently corrupted. This is likely for large non-ASCII logs with the default 256 KiB page size; align windows to character boundaries or carry decoder state between pages.
Useful? React with 👍 / 👎.
These were the only changes in this branch that touched agent-core. They persisted two previously live-only signals (step retries, tool progress) to the wire purely so the visualizer could display them — marginal features that did not justify modifying the core loop or extending the wire surface. Reverts the agent-core loop/type/export changes (restored to main, keeping #1209) and its changeset, and removes the vis-side rendering and types that consumed step.end.retries / tool.result.progress. The rest of vis is unchanged and reads only data agent-core already persists.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b524fee753
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (agents.length === 0) { | ||
| agents = await discoverAgentsFromDisk(sessionDir); |
There was a problem hiding this comment.
For an imported zip whose readable state.json.agents contains some stale entry but omits main, the import still succeeds because agents/main/wire.jsonl is present, but this condition skips disk discovery and detail.agents lacks main; the default Wire/Timeline requests then 404 for an otherwise valid bundle. The current fallback only runs when inventoryAgents returns an empty array, so partial imported inventories still need to be merged with disk discovery.
Useful? React with 👍 / 👎.
1 participant
Make
apps/vissurface everything a kimi-code session persists, and turn it from a record viewer into an analysis tool.What changed
1. Background tasks & cron (
feat(vis): surface background tasks and cron jobs)The visualizer read every wire/state/blob artifact but ignored the two on-demand families agent-core writes per session — neither reconstructable from the wire:
tasks/<id>.json+output.log(process / agent / question kinds) with status, timing, kind-specific fields, and a progressively paged log viewer (byte-window paging via an exactnextOffsetcursor).cron/<id>.json(expression, prompt, recurring/one-shot, last-fired).2. agent-core: persist step retries and tool progress (
feat(agent-core): …)Two transient signals were live-only, so nothing survived for post-hoc analysis. Both are additive optional wire fields — no protocol bump, existing records keep loading:
step.end.retries— recovered transient provider failures (via a newonRetrycallback onchatWithRetry).tool.result.progress— a bounded sparse-progress summary (updateCount/lastStatus/maxPercent); streamed stdout/stderr is deliberately excluded.LoopStepRetryRecord,LoopToolProgressSummary. Includes a changeset.3. vis: execution-analysis timeline (
feat(vis): add execution-analysis timeline …)New Timeline tab folds the wire into turns → steps → tool calls client-side and derives what the flat list hides: per-turn/step/tool durations, per-turn token cost, a context-window-fill sparkline + cache-hit rate, tool usage stats, idle-gap detection, and a config-change timeline. Inline: Wire rows show tool call→result elapsed time and result truncation/size/retries/progress; the Issues drawer gains tool-error / truncation / filtered / max_tokens / retried categories; Tasks links agent tasks to the subagent's wire.
Tests