sanad: worker runtime p0 — walking skeleton (deploy, invoke, trace, pause)

[osodev-1]

What this is

The worker-runtime P0 walking skeleton — deploy a worker-agent bundle, invoke it synchronously as an ephemeral budgeted run, stream its journal, persist its trace, pause it. Spec and plan live on worker-agents-prd: docs/superpowers/specs/2026-08-13-worker-runtime-design.md + docs/superpowers/plans/2026-08-13-worker-runtime-p0-walking-skeleton.md.

Locked decisions this implements: run-per-invoke (ephemeral soul, fresh session = the trace); machine per (workspace, env) reusing the Fargate wake machinery, subprocess-per-run; DB run rows + S3 traces via presigned URLs; one accountable owner per agent (deploy-blocked without one); ReturnOutput tool as the interface contract (nudge locally, fail-fast no_output in cloud); budgets clamped machine-side.

The three stacks

sanad-web — worker tables (workspaces/agents/versions/deployments/runs/invoke tokens/machines) + migrations 0006–0007; itok_ invoke tokens (quota at mint); agent registry with stable-ownership upsert, deployment supersede rule, version-ancestry check; per-(workspace,env) machine wake with boot-compatible task env + keep_warm; sync invoke route (NDJSON passthrough, idempotency keys, 120s wake bound, honest 4xx passthrough, storage_unconfigured pre-wake guard); completion ingest with pricing rollup; status-guarded lost-run reaper with maintained staleness signal; runs read APIs; minimal /agents pages + per-agent OpenAPI.

kimi CLI — kimi_cli/worker/ assembly (worker.yaml sidecar, derived agent spec preserving base tools, ReturnOutput stop-turn tool shared verbatim by both runtimes); sanad agent dev (local ephemeral run, exit-code contract 0/1/3/4); sanad agent deploy/runs/logs/pause/resume against the real route contracts.

terminal-server — RunRunner (one-turn wire subprocess, token budget with status-guarded trip, defensive telemetry parsing); gated /internal/worker/* routes (fail-closed WORKER_ENABLED, bundle containment hardened against traversal/degenerate/conflicting keys, budget clamps, symmetric spawn cleanup); gzip trace upload via presigned PUT + completion report + registry drop; keep-warm idle probe.

Parity evidence (DX-4): tests_e2e/test_worker_parity.py runs the same bundle through sanad agent dev and the real cloud RunRunner — identical output document, full event sequence observed, mutation-tested non-vacuous.

Tests

540 passing on this exact tree: 306 sanad-web (vitest, incl. contract tests for the invoke/complete/openapi routes), 201 terminal-server, 33 CLI worker + e2e (dev + parity). Every task landed through TDD with an independent review + fix loop; a final whole-branch review (cross-task seams, security posture, run lifecycle) drove one last hardening wave (reaper staleness maintenance, guarded run-state transitions, idempotency-key clearing on infra failures, machine-4xx passthrough).

Before first staging deploy

• Create the S3 bucket + set SANAD_RUNS_BUCKET; grant the sanad-workspace-task role PUT (and the control plane GET) on runs/* — unset bucket now fails closed pre-wake with storage_unconfigured.
• Set CRON_SECRET and schedule POST /api/internal/cron/reap-runs.
• No new machine image needed — the terminal-server task image gains worker mode via WORKER_ENABLED=1/KEEP_WARM injected at task launch.

Sign-off wanted

• MODEL_PRICING numbers are placeholders (kimi-k3: $0.60/MTok in, $2.50/MTok out) — flag before anything billing-visible.

Fast-follows (tracked in the branch ledger, none merge-blocking)

Drizzle meta snapshots 0003–0005 backfill (chip open); test-strengthening (supersede WHERE assertion, outside-work-dir bundle case, missing-agent.yaml case); live_run_ids() accessor instead of _runs reach-in; keepWarm live-flip to a running machine; per-wake ensureAccessPoint latency; /agents Nav link; explicit KIMI_DISABLE_TELEMETRY in the e2e env; the 7 auto-fixable pre-existing ruff errors blocking repo-wide make check; storage_unconfigured keeps its idempotency key (revisit with retry UX).