Skip to content

Commit 7c7aa09

Browse files
tombeckenhamclaudeautofix-ci[bot]AlemTuzlak
authored
feat: resumable streams — reconnect to in-flight SSE responses via pluggable delivery durability (#955)
* feat: resumable SSE streams via pluggable delivery durability Add a transport-level StreamDurability seam to toServerSentEventsResponse: chunks are appended to an ordered log before delivery and each SSE event is tagged with an opaque adapter-owned id: offset. Reconnects (Last-Event-ID) and joins (?offset=-1&runId) replay from the log without re-running the provider. Ships memoryStream (in-core, dev/test) and the new @tanstack/ai-durable-stream package (Durable Streams protocol adapter). Client: fetchServerSentEvents now auto-resumes id-tagged streams, de-dupes replayed prefixes, exposes joinRun(runId), and throws DurableStreamIncompleteError when a durable run ends with no terminal event and no forward progress. Split out of #785 so state persistence and delivery durability land independently. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A6Arc9bWdLq1aRRnDRCLMt * docs(resumable-streams): Cloudflare Durable Streams backend via service binding Document running against the Durable Streams Cloudflare Workers + DO backend: same protocol, no new adapter — inject the service binding's fetch via the adapter's injectable fetch option, or point server at the deployed Worker URL. Note the DO alarm satisfies the lease/reaper needed for producer-death terminalization. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A6Arc9bWdLq1aRRnDRCLMt * docs(durable-stream): Cloudflare service-binding example; make server optional when fetch is provided The durableStream adapter is a protocol client, so a Cloudflare Workers + Durable Objects backend that speaks the same protocol needs no new adapter — just the injected `fetch` seam. Over a service binding the host is irrelevant (dispatch routes to the bound Worker by path), so `server` is now optional whenever `fetch` is supplied and defaults to a reserved `.internal` base. Passing neither `server` nor `fetch` throws loudly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(resumable-streams): bound eviction, reconnection, and surface silent failures Addresses review findings on the resumable-streams PR: #1 memoryStream never evicted its process-global log Map (unbounded growth). Completed logs are now swept after a grace window with a hard LRU cap; active runs are never evicted. Adds MemoryStreamOptions. #3 memoryStream join/resume of an unknown or evicted run parked forever. A concrete resume of a missing log now throws; a from-start join bounds the wait for the first chunk (firstChunkDeadlineMs) instead of hanging. #2 The client resumable-SSE reconnect loop and the durableStream read loop were unbounded and backoff-free. The client now throttles between attempts and caps the total (StreamReconnectLimitError); durableStream caps consecutive body-read-failure retries. Normal long-poll advancement is never throttled. Adds reconnect options to both. #4 Durability terminal-append / close failures are rethrown to the live consumer but invisible to a replaying joiner. toServerSentEventsResponse now accepts `debug` to record the real cause server-side via the library's logger. Also: durableStream `server` is optional when `fetch` is provided (service bindings). Docs + changeset updated; unit tests added for each path (timing- and eviction-based behavior is covered by unit tests rather than the aimock e2e harness, which can't exercise it deterministically). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: apply automated fixes * docs(example): add resumable-streams demo to ts-react-chat New /resumable route pair: api.resumable.ts (memoryStream-backed POST that appends+tags each SSE event, plus a GET joinRun replay endpoint) and resumable.tsx (start a run, then join it by run ID — in a second tab or after a reload — replaying from the durability log without re-running the model). Nav link added to Header. Kept the shared api.tanchat route untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(resumable-streams): note other durability backends (Electric, etc.) Clarify that durableStream works with any Durable Streams protocol server, and that other systems (a Postgres-backed log via Electric, Redis streams, a queue) can back durability by implementing the four-method StreamDurability interface. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai, ai-client): resumable NDJSON + XHR delivery durability Extend resumable streams (delivery durability) beyond SSE to NDJSON and the XHR transports. Server (@tanstack/ai): - toHttpStream gains an optional getId; when present each NDJSON line is emitted as an { id, chunk } envelope (NDJSON has no native event id). Untagged streams stay bare lines, byte-identical to before. - toHttpResponse gains durability/batch/debug, reusing the same durableStreamSource as toServerSentEventsResponse. Client (@tanstack/ai-client): - Generalize the SSE-only reconnect loop into transport-agnostic resumableStream(openEventSource, signal, reconnect). Shared line parsers (linesToSSEEvents/linesToNdjsonEvents) feed fetch (fetchEventSource) and XHR (xhrEventSource) thunks. - fetchHttpStream, xhrServerSentEvents, xhrHttpStream are now resumable and expose joinRun. XHR onerror surfaces StreamReadError so a durable XHR run can reconnect; StreamReadError message is now transport-neutral. Tests: NDJSON server durability suite, NDJSON+XHR resumable-transport client suite, NDJSON arm on the delivery-durability e2e harness + spec. Docs/skill/changeset updated for NDJSON + XHR. * fix(ai, ai-client): CR round 1 — durability logging, memory-log leak, CRLF, [DONE] id parity Round 1 review fixes (7-agent CR): - HIGH: toServerSentEventsResponse / toHttpResponse constructed the durability logger only when debug was passed, so terminal-append/close failures were silently swallowed by default (fully lost on the client-disconnect path). Now instantiate resolveDebugOption(debug) unconditionally, matching every other activity (errors category is on by default). - HIGH: memoryStream.read() called getOrCreateLog before the unknown-run check, leaving a permanent empty log per unknown/evicted resume — unbounded growth defeating the eviction logic. Peek with memoryLogs.get; a concrete offset for an absent run throws without inserting; a from-start join creates the log for the produce race but deletes it on the first-chunk deadline. - readStreamLines (fetch) now strips a trailing CR, matching readXhrLines, so CRLF SSE servers do not miss the [DONE] sentinel. - Fetch SSE [DONE] synthesis now threads the run's ids (parity with the XHR xhrSSEParser), so a [DONE]-terminating server that omits ids still yields a correlated terminal. - Clarified the ReconnectOptions.maxAttempts comment (counts total lifetime reconnects, not consecutive no-progress ones). - Softened the changeset claim: completion terminalizes when the source emits its own terminal event. - SKILL.md anti-pattern examples: gpt-4o -> gpt-5.5. - Tests: fetch NDJSON reconnect test uses reconnect delayMs:0; parseNdjsonEvents helper mirrors the production !('type' in value) envelope guard. Call sites cleared: - responseToSSEEvents: added optional 3rd param fallbackIds; all existing callers (responseToSSEChunks, fetch connect/joinRun) pass <=3 args, backward compatible. - readStreamLines / memoryStream.read: signatures unchanged; behavior-preserving except the removed phantom insertion (observable throw/reject paths unchanged, already covered by stream-durability.test.ts). * fix(ai-client): CR round 2 — reconnect resilience + joinRun id parity Round 2 confirmation-round fixes: - resumableStream: a transport drop (StreamTruncatedError/StreamReadError) now retries whenever an offset is held, even if THAT attempt made no new progress (a caught-up run whose parked long-poll socket drops, or a proxy that drops right after replaying the de-duped overlap). The total-attempts ceiling still bounds a genuine flapper; the per-attempt progress requirement only converted recoverable drops into hard failures on flaky networks. The clean-end path stays strict and now documents the invariant it relies on (a durable transport must not surface an empty long-poll window as a clean end; both shipped backends honor it). - xhrServerSentEvents.joinRun now threads { runId } into the [DONE] fallback, matching fetchServerSentEvents.joinRun (correlation parity). - e2e parseNdjson + toHttpResponse @param prose aligned (envelope guard; batch is nested under durability, debug documented). - docs: durable sources must emit their own terminal; memoryStream is for replaying completed runs (live mid-stream resume needs a backend whose producer outlives the delivery socket); qualified producer-death headline as backend-driven. Covering test: a reconnect that replays only the de-duped overlap then drops is retried, not surfaced as an error (connection-adapters-resumable.test.ts). Call sites cleared: resumableStream catch condition — only relaxed the retry guard (dropped '&& progressed'), kept StreamReadError/StreamTruncatedError type gate + lastEventId gate, so a first-attempt failure with no offset still rethrows (asserted by existing 'does not retry HTTP setup failures' test). * fix(ai, ai-client): CR round 3 — producer robustness, fetch retry, NDJSON headers, docs Round 3 confirmation-round fixes (scope widened per request to cover the pre-existing durability-producer bugs). Producer (durableStreamSource): - Flush buffered-but-unflushed chunks to the log before terminalizing on the abort/disconnect path (previously up to batchSize-1 already-produced chunks were dropped, so a joiner replayed a truncated prefix). Matches the error path. - Prefer the real provider error over a generic AbortError when a run both fails and is aborted, so a joiner sees the true cause. - Do not rethrow a post-terminal close()/append failure to the live consumer once a terminal was already forwarded — rethrowing appended a contradictory RUN_ERROR after RUN_FINISHED on the wire. Late cleanup failures are recorded server-side via logger.errors instead. - validateOffset now also rejects offsets with surrounding whitespace (the SSE client .trim()s the id, so such an offset would not round-trip on reconnect). Client: - normalizeConnectionAdapter.send: guard terminal synthesis in the catch so a missing-id throw can't mask the original error. - fetchEventSource: wrap a fetch() rejection (offline/DNS/refused) as StreamReadError so a reconnect retries from the offset, matching XHR; a first-attempt failure with no offset still surfaces. - readStreamLines: final decoder.decode() flush so a cut mid-multibyte-char is reported as truncation. Server transport: - toHttpResponse defaults Content-Type to application/x-ndjson + no-cache (overridable), matching the SSE helper, so intermediaries don't buffer it. Docs/skill/harness: - JSDoc @examples use openaiText('gpt-5.5') (chat has no model field) and wrap durability examples in a POST handler; model ids normalized to gpt-5.5 across connection-adapters.md + SKILL.md; SKILL sources += resumable-streams; doc reconnection wording scoped to the clean-end path; e2e X-Run-Id no longer advertised on reconnect; harness + seen-set comments. Tests: flush-on-abort, double-terminal-suppression, whitespace-offset rejection, NDJSON Content-Type, and fetch-rejection retry (+ first-attempt surfacing). Deferred (low, out of delta subject): fetch body not cancelled on early terminal return (reverted — the reader-cancel broke mock-reader teardown in ~26 pre-existing tests; XHR-parity nit, durable backends close on terminal anyway). * fix(ai-client, docs): CR round 4 — doc-accuracy, comment precision, coverage Round 4 confirmation-round fixes (docs/comments/tests + one defensive guard; no new production logic bugs were found this round). Docs (correcting inaccuracies introduced in earlier CR rounds): - Reconnection-bounding section now states the durable-vs-non-durable distinction accurately: a transport error retries while an offset is held; a durable clean end with no progress fails with DurableStreamIncompleteError; only a non-durable clean end is a completed run. Documents why the asymmetry is deliberate. - connection-adapters.md no longer groups xhrServerSentEvents (SSE) under the NDJSON/toHttpResponse sentence. - Added a reconnect-safety warning: the client auto-reconnects by re-POSTing, so non-idempotent POST-handler work must be guarded behind a resume check. - config.json: dropped the redundant updatedAt on the newly-added overview page. - changeset: reconnect option applies to all four HTTP adapters, not just fetchServerSentEvents. Code: - readXhrLines.finish() now guards status===0 like enqueueDelta (avoids a bogus 'status: 0' error if loadend fires before load/error/abort). - Comments: linesToSSEEvents one-id-per-data-event assumption; clarified the fetch-rejection wrap note. Tests: - XHR onerror→reconnect (proves StreamReadError from onerror drives a retry with Last-Event-ID) and NDJSON provider-throw terminal persistence — closing the highest-value coverage gaps on the XHR/NDJSON surface. - delayMs:0 on the reconnecting fetch-SSE tests (speed/consistency). Deferred (pre-existing / by-design / out-of-delta-subject): reconnect clean-end asymmetry (correct + documented), abortableIterable listener cleanup, fetch body cancel on early exit (reverted — broke mock-reader teardown), pump finally-throw surfacing, SSE persistent-id interop. * fix(ai, ai-client, docs): CR round 5 — self-inflicted doc/comment staleness + one silent-swallow Round 5 confirmation-round fixes. No genuine code-logic defects surfaced this round; the items below are (a) one silent failure introduced by the R3 double-terminal guard and (b) doc/comment staleness introduced by earlier rounds, plus a pre-existing doc-example hang bug. Code: - durableStreamSource: a producer error thrown AFTER a terminal was forwarded was suppressed by the !terminalForwarded rethrow guard (correct — avoids a contradictory second terminal) but never logged, so it vanished. Now logged via logger.errors like the close/terminal-append failures. Covering test added. Docs/comments (correcting staleness from earlier rounds): - debug JSDoc (both response helpers) + overview.md prose no longer imply server-side logging requires ; the errors category is on by default (R1 change), and debug only routes/raises verbosity. - toServerSentEventsStream JSDoc documents its getId param (parity w/ toHttpStream). - overview.md GET join example guards a missing offset and its comment no longer over-claims 'never iterates the provider' for a bodyless produce path. - Removed review-artifact comments ('Finding 6', 'the R1 comment claims'). Docs (pre-existing example bug, flagged twice): - WebSocket subscribe() example drains the queue before honoring (a burst + close in one macrotask previously dropped queued chunks, incl. a trailing RUN_FINISHED → client hang) and registers the abort listener once. Deferred to a follow-up (pre-existing / off NDJSON-XHR subject / documented design): SSE heartbeat/empty-data frame tolerance; abortableIterable orphan- promise .catch; joinRun offset=-1 + Last-Event-ID precedence; reconnect lifetime-ceiling on healthy socket-per-event runs; fetch body-cancel on early terminal; assorted test-hygiene (shared FakeXhr, delayMs). * feat(ai-client, docs): reconnect ceiling = consecutive-no-progress (default 5); custom-adapter guide Addressing review feedback: - Reconnect ceiling: maxAttempts now bounds CONSECUTIVE reconnects that deliver no new events (default lowered 1000 -> 5); forward progress resets the counter. A healthy long run (even a socket-per-event proxy) never approaches it; it fires only when the run is genuinely stuck. This also resolves the CR finding that the old total-lifetime ceiling could fail a healthy progressing run. Ceiling test split into a no-progress-flapper (hits it) + a progress-resets test (does not). - stream-to-response: terminalForwarded lint fix (scoped no-unnecessary-condition disable; the flag is only assigned inside the flush() closure that TS CFA cannot observe). Docs: - New guide docs/resumable-streams/custom-adapter.md: implement the four-method StreamDurability contract over your own store, the offset/park/terminalize rules, wiring, and offset branding. Registered in config.json, cross-linked from the overview. - chat/connection-adapters: show the GET handler (joinRun) alongside POST. - overview reconnection-bounding section updated to the new semantics + default. NOTE: did NOT make memoryStream a silent default (explored per request, then reverted on review) - durability stays opt-in to avoid shipping an in-process, single-process-only, per-run-buffering backend to production by default. advertiseRunId is a local var in the e2e harness route, not public API. * fix(ai-client, ai-durable-stream): address CodeRabbit review feedback - SSE id parsing: preserve the opaque offset verbatim (strip only a single leading space per the SSE spec, no trim, which would mangle a valid offset), and treat an empty id: as a resume-cursor reset (drop lastEventId + clear the de-dupe set) rather than a durable empty offset. - resolveReconnectOptions: reject non-finite / negative maxAttempts and delayMs up front so a NaN/Infinity ceiling cannot cause unbounded reconnects. - durable-stream read: throw on non-strictly-increasing record sequences within a response instead of silently dropping later records. - durable-stream: new operationTimeoutMs (default 30000) bounds create/append/ close via an AbortSignal so a stalled backend cannot hang delivery or terminalization; long-poll reads are intentionally excluded. - e2e delivery-durability spec: document the aimock-policy exemption. Tests added: empty-id reset + invalid-reconnect-bounds (ai-client), non-monotonic seq rejection + operation timeout (ai-durable-stream). Not applied (verified against the code): peer-dep workspace:^ is consistent with all sibling packages (changing to * would break sherif); memory retention is already bounded (sweepMemoryLogs + TTL + cap); the throw-in-finally is intentional aggregation and ESLint-suppressed (repo does not use Biome); batch is already documented as nested; tests already follow the package tests/ dir convention. * ci: apply automated fixes * docs(resumable-streams): simplify overview to the happy path, split advanced out - overview.md: rewritten as the 3-step common case (pick an adapter, wrap the response with POST+GET, client is automatic). Removed em dashes. No longer makes it look harder than it is. - advanced.md (new): moved the deep material here — durableStream options, joinRun (attach-by-id), completion/stop/errors, memoryStream-in-production, reconnection bounding, offset ownership, Cloudflare, process death, and delivery-is-not-state. - joinRun is now documented under Advanced (it is a manual, opt-in API; the common reconnect-on-drop path needs no client code). - Scrubbed em dashes from custom-adapter.md and the resumable sections I added to connection-adapters.md; fixed the custom-adapter process-death link to point at the advanced page. - config.json: registered the Advanced page. * docs(resumable-streams): drop dead chat() placeholder from GET replay handler The resume path serves entirely from the durability log and never iterates the source stream, so the chat() call and its replay: threadId were dead code. Replace with an empty stream and guard that offset is present so a bare GET does not fall through to the produce path. * feat(ai): add resumeServerSentEventsResponse / resumeHttpResponse helpers A resume GET is served entirely from the durability log and never iterates a producer stream, so the response helpers previously forced callers to fabricate an empty stream in every GET handler. These helpers take just the durability adapter, do the replay, and return a 400 when the request carries no resume offset. Dogfood them in the e2e harness and the example app, and simplify the docs GET handler to a one-liner. * ci: apply automated fixes * docs(chat): use resumeServerSentEventsResponse in the connection-adapters GET example The resumable-SSE server example still constructed a dead chat() with a replay: threadId in its GET handler. Replace it with the resume helper. * refactor(example): resumable demo uses useChat; docs show the reconnect side-effect guard Rewrite the ts-react-chat resumable route from hand-rolled connection driving (useState/useRef/drainInto + manual connect/joinRun) to a plain useChat page, matching the overview doc: the durable route makes reconnect automatic with no client code. Expand the overview reconnect gotcha into a runnable example that guards one-time side effects behind durability.resumeFrom(). * ci: apply automated fixes * fix(ai-client): don't truncate agentic runs at the first RUN_FINISHED The resumable stream engine returned on the first RUN_FINISHED/RUN_ERROR. An agent loop emits one RUN_STARTED/RUN_FINISHED pair per turn, so a tool-calling run carries several terminals in a single response — returning on the first dropped the tool result and the final answer. This engine drives every stream (durable and non-durable alike), so it regressed existing non-durable clients: all tool/agentic/custom-event/structured E2E tests hung after the first turn while plain single-turn streams passed. Drain the event source to its natural end (the server closes the response only when the run is truly complete) and use the terminal flag post-loop to decide done-vs-reconnect. Restores the pre-durability read-to-close behavior for non-durable streams; durable behavior is unchanged (single-terminal responses end right after the terminal, so every resumable unit test still holds). * fix(ai-client): send durability run id as X-Run-Id header, not a query param The resumable adapters appended `?runId=<id>` to every POST (useChat always supplies a runId), rewriting the request URL for all existing clients — not just durable ones. That broke callers/tests that match the bare endpoint URL and violated the invariant that a non-durable request is byte-identical to a plain fetch. Send the client-chosen run id in an `X-Run-Id` request header instead. The POST URL is now untouched, so existing clients are unaffected, while a durability sink still keys its log by the client's run id (memoryStream's readRunId reads the header first, then falls back to the `?runId` query the GET join path still uses). Durability remains a transparent add-on. * ci: apply automated fixes --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Alem Tuzlak <t.zlak@hotmail.com>
1 parent 54a51fa commit 7c7aa09

35 files changed

Lines changed: 7510 additions & 278 deletions

.changeset/resumable-streams.md

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
---
2+
'@tanstack/ai': minor
3+
'@tanstack/ai-client': minor
4+
'@tanstack/ai-durable-stream': minor
5+
---
6+
7+
Resumable streams: reconnect to an in-flight SSE **or NDJSON** response without
8+
re-running the provider.
9+
10+
`toServerSentEventsResponse` and `toHttpResponse` both accept a
11+
`durability: { adapter, batch }` option. The adapter (`StreamDurability`)
12+
records every chunk to an ordered log before delivery and tags each event with
13+
an opaque, adapter-owned offset — an SSE `id:` line, or the `id` of an NDJSON
14+
`{ id, chunk }` envelope (NDJSON has no native event-id). A reconnect
15+
(`Last-Event-ID`) or an explicit `?offset` read replays strictly after that
16+
offset from the log — the lazy provider stream is never iterated on resume.
17+
Producers terminalize the log on cancellation and failure (`RUN_ERROR` append
18+
19+
- `close()`) and on completion when the source stream emits its own terminal
20+
event (`chat()` always does), so readers are never parked on a dead run.
21+
22+
Two adapters ship: `memoryStream(request)` in `@tanstack/ai` (process-local,
23+
for development and tests) and the new `@tanstack/ai-durable-stream` package,
24+
a Durable Streams protocol adapter for production backends.
25+
26+
For the `GET` handler that a reload or a second tab reconnects to,
27+
`resumeServerSentEventsResponse({ adapter })` and `resumeHttpResponse({ adapter })`
28+
replay a run straight from the durability log. They need no producer stream and
29+
return a 400 when the request carries no resume offset.
30+
31+
On the client, all four HTTP adapters are now resumable — `fetchServerSentEvents`,
32+
`fetchHttpStream`, `xhrServerSentEvents`, and `xhrHttpStream`. Each tracks the
33+
per-event offset, auto-reconnects with `Last-Event-ID`, de-duplicates the
34+
replayed prefix, and exposes `joinRun(runId)` to attach to an in-flight or
35+
finished run from the start (read-only GET with `offset=-1`). Untagged streams
36+
behave exactly as before. A durable run that ends with no terminal event and no
37+
forward progress now throws `DurableStreamIncompleteError` instead of hanging.
38+
39+
Reconnection and durability are bounded so failures surface rather than hang or
40+
loop:
41+
42+
- `memoryStream` evicts completed logs after a grace window (unbounded growth
43+
is gone); resuming an expired/unknown run throws, and a from-start join to a
44+
run that never produces fails after `MemoryStreamOptions.firstChunkDeadlineMs`.
45+
- all four HTTP adapters accept `reconnect: { maxAttempts, delayMs }` — a
46+
throttle plus a ceiling on CONSECUTIVE no-progress reconnects (default 5;
47+
forward progress resets it) that fails with the new `StreamReconnectLimitError`
48+
instead of reconnecting endlessly, without penalizing a healthy long-lived run.
49+
- `durableStream` accepts `reconnect: { maxReadFailures, delayMs }` to bound its
50+
read-retry loop, and `server` is now optional when `fetch` is provided (e.g. a
51+
Cloudflare service binding).
52+
- `toServerSentEventsResponse` accepts `debug` to record durability terminal /
53+
close failures server-side, where a replaying joiner cannot observe them.

docs/chat/connection-adapters.md

Lines changed: 84 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -73,13 +73,65 @@ import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";
7373

7474
const { messages } = useChat({
7575
connection: fetchServerSentEvents("/api/chat", {
76-
body: { provider: "openai", model: "gpt-5.1" },
76+
body: { provider: "openai", model: "gpt-5.5" },
7777
}),
7878
});
7979
```
8080

8181
> **Tip:** `body` and `forwardedProps` populate the same wire field. Use `body` for static defaults, the `forwardedProps` constructor option (or per-`sendMessage` `data`) for dynamic values. Runtime values always win.
8282
83+
### Resumable SSE
84+
85+
`fetchServerSentEvents` watches SSE `id:` values. If a connection drops after
86+
receiving an id, it reconnects with `Last-Event-ID` and de-duplicates the
87+
replayed prefix. `joinRun(runId)` performs a read-only GET with `offset=-1` and
88+
the run id, replaying an in-flight or finished run from the start.
89+
90+
The ids only appear when the server passes a durability adapter to
91+
`toServerSentEventsResponse`. They are opaque tokens owned by that adapter; the
92+
chat client does not create, parse, or persist them. Without ids, behavior is
93+
identical to a plain single fetch. See
94+
[Resumable Streams](../resumable-streams/overview).
95+
96+
Your route needs a `GET` handler alongside `POST` for `joinRun` (second tab or
97+
reload) to work. `POST` handles fresh runs and auto-reconnects (it re-sends the
98+
same body with `Last-Event-ID`); `GET` replays a known run from the start:
99+
100+
```typescript
101+
import {
102+
chat,
103+
chatParamsFromRequest,
104+
memoryStream,
105+
resumeServerSentEventsResponse,
106+
toServerSentEventsResponse,
107+
} from "@tanstack/ai";
108+
import { openaiText } from "@tanstack/ai-openai";
109+
110+
export async function POST(request: Request) {
111+
const { messages, threadId, runId } = await chatParamsFromRequest(request);
112+
const stream = chat({ adapter: openaiText("gpt-5.5"), messages, threadId, runId });
113+
return toServerSentEventsResponse(stream, {
114+
durability: { adapter: memoryStream(request) },
115+
});
116+
}
117+
118+
// joinRun hits GET ?offset=-1&runId=... (replay only, no messages sent).
119+
export async function GET(request: Request) {
120+
return resumeServerSentEventsResponse({ adapter: memoryStream(request) });
121+
}
122+
```
123+
124+
The `GET` handler calls no provider: on a replay the durability adapter's
125+
`resumeFrom()` is non-null (from `?offset`), so the log is replayed instead.
126+
`resumeServerSentEventsResponse` returns a 400 when the request has no resume
127+
offset. Use `resumeHttpResponse` for the NDJSON adapters.
128+
129+
`fetchHttpStream` and `xhrHttpStream` resume the same way over NDJSON, where the
130+
offset rides in an `{ id, chunk }` envelope (see below) instead of an SSE `id:`
131+
line. Enable it by passing a durability adapter to `toHttpResponse`.
132+
`xhrServerSentEvents` resumes over SSE exactly like `fetchServerSentEvents`
133+
(paired with `toServerSentEventsResponse` and its `id:` lines).
134+
83135
## HTTP Streaming (NDJSON)
84136

85137
For environments that don't speak SSE — some edge runtimes, certain mobile WebViews, or anywhere a proxy strips `text/event-stream` — use raw newline-delimited JSON. The wire format is one JSON `StreamChunk` per line:
@@ -92,7 +144,9 @@ const { messages } = useChat({
92144
});
93145
```
94146

95-
Server-side, write each chunk as `JSON.stringify(chunk) + "\n"` to the response body. Options (`url`, `headers`, `body`, `fetchClient`, dynamic functions) match `fetchServerSentEvents` exactly.
147+
Server-side, write each chunk as `JSON.stringify(chunk) + "\n"` to the response body (or use `toHttpResponse(stream)`). Options (`url`, `headers`, `body`, `fetchClient`, dynamic functions) match `fetchServerSentEvents` exactly.
148+
149+
`fetchHttpStream` is also resumable: pass a durability adapter to `toHttpResponse` and each line becomes an `{ id, chunk }` envelope. A dropped connection reconnects with `Last-Event-ID`, de-duplicates the replayed prefix, and `joinRun(runId)` attaches to an existing run. Same guarantees as [Resumable SSE](#resumable-sse), over NDJSON.
96150

97151
## React Native and Expo
98152

@@ -129,6 +183,10 @@ const chat = useChat({
129183
});
130184
```
131185

186+
Mobile connections drop often, so this is where resumability pays off most.
187+
Both XHR adapters reconnect and `joinRun` when the server adds a durability
188+
adapter. See [Resumable Streams](../resumable-streams/overview).
189+
132190
Use `xhrServerSentEvents()` when your server returns `text/event-stream` via
133191
`toServerSentEventsResponse()`:
134192

@@ -203,7 +261,7 @@ export const chatFn = createServerFn({ method: "POST" })
203261
.inputValidator((data: { messages: Array<UIMessage> }) => data)
204262
.handler(({ data }) =>
205263
toServerSentEventsResponse(
206-
chat({ adapter: openaiText("gpt-5.1"), messages: data.messages }),
264+
chat({ adapter: openaiText("gpt-5.5"), messages: data.messages }),
207265
),
208266
);
209267
```
@@ -292,20 +350,30 @@ function websocketConnection(url: string): SubscribeConnectionAdapter {
292350

293351
return {
294352
async *subscribe(abortSignal) {
295-
while (!abortSignal?.aborted && !closed) {
296-
const buffered = queue.shift();
297-
if (buffered !== undefined) {
298-
yield buffered;
299-
continue;
300-
}
301-
const chunk = await new Promise<StreamChunk | null>((resolve) => {
302-
pending = resolve;
303-
abortSignal?.addEventListener("abort", () => resolve(null), {
304-
once: true,
353+
// Register the abort listener once (not per-iteration) so it can't
354+
// accumulate on a long-lived socket.
355+
const onAbort = () => deliver(null);
356+
abortSignal?.addEventListener("abort", onAbort, { once: true });
357+
try {
358+
while (!abortSignal?.aborted) {
359+
// Drain buffered chunks BEFORE honoring `closed`: a burst of messages
360+
// followed by a close event (common within one macrotask) must still
361+
// deliver the queued chunks (including a trailing RUN_FINISHED),
362+
// otherwise the client would hang waiting for a terminal it dropped.
363+
const buffered = queue.shift();
364+
if (buffered !== undefined) {
365+
yield buffered;
366+
continue;
367+
}
368+
if (closed) return;
369+
const chunk = await new Promise<StreamChunk | null>((resolve) => {
370+
pending = resolve;
305371
});
306-
});
307-
if (chunk === null) return;
308-
yield chunk;
372+
if (chunk === null) return;
373+
yield chunk;
374+
}
375+
} finally {
376+
abortSignal?.removeEventListener("abort", onAbort);
309377
}
310378
},
311379

docs/config.json

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,8 @@
167167
{
168168
"label": "Connection Adapters",
169169
"to": "chat/connection-adapters",
170-
"addedAt": "2026-04-15"
170+
"addedAt": "2026-04-15",
171+
"updatedAt": "2026-07-17"
171172
},
172173
{
173174
"label": "Thinking & Reasoning",
@@ -181,6 +182,26 @@
181182
}
182183
]
183184
},
185+
{
186+
"label": "Resumable Streams",
187+
"children": [
188+
{
189+
"label": "Overview",
190+
"to": "resumable-streams/overview",
191+
"addedAt": "2026-07-17"
192+
},
193+
{
194+
"label": "Advanced",
195+
"to": "resumable-streams/advanced",
196+
"addedAt": "2026-07-17"
197+
},
198+
{
199+
"label": "Custom Durability Adapter",
200+
"to": "resumable-streams/custom-adapter",
201+
"addedAt": "2026-07-17"
202+
}
203+
]
204+
},
184205
{
185206
"label": "Protocol",
186207
"children": [

0 commit comments

Comments
 (0)