Skip to content

Commit 0de8e83

Browse files
tannerlinsleytannerlinsley
authored
docs(research): ai-orchestration integration memo (#7)
docs(research): ai-orchestration integration memo
2 parents c98f260 + 76bd7b1 commit 0de8e83

2 files changed

Lines changed: 278 additions & 8 deletions

File tree

‎research/AI_ORCHESTRATION_INTEGRATION.md‎

Lines changed: 269 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,269 @@
1+
# `@tanstack/ai-orchestration` ↔ `@tanstack/workflow-core` integration
2+
3+
What can be reliably removed from [TanStack/ai#542](https://github.com/TanStack/ai/pull/542) now that `@tanstack/workflow-core@0.0.1` is live, and how the AI APIs there compose with the shipping engine.
4+
5+
PR head as of writing: `3a51d7b`.
6+
7+
## TL;DR
8+
9+
**Path B — refactor `ai-orchestration` to sit on top of `@tanstack/workflow-core`.** Sheds 18 of 25 source files; the AI layer narrows to: `defineAgent`, `defineOrchestrator`, `defineRouter`, the `invokeAgent` shape-detector helper, an `agentTypes`/`types.ts` for AI-only declarations, and (optional) a `withAgents` middleware that adds `ctx.agent(stepId, def, input)` for ergonomic invocation.
10+
11+
The shape difference is real: PR 542 ships a generator-based engine (`async function*` + `yield* agents.x(input)` + `StepDescriptor` union); workflow-core ships a closure engine (`async (ctx)` + `await ctx.step('id', fn)`). Closure-based AI integration is the simpler direction.
12+
13+
**No workflow-core engine changes are strictly required** — closure-over-`ctx` is enough for step bodies to forward agent streams via `ctx.emit`. One small affordance (a typed `ctx.agent` middleware) would polish the DX but isn't a blocker.
14+
15+
## The shape mismatch
16+
17+
| | `ai-orchestration` today | `workflow-core` |
18+
| --------------------- | ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
19+
| Workflow function | `async function* run({ input, state, agents, emit, signal })` | `async (ctx) => ...` |
20+
| Side-effect primitive | `yield* step('id', fn)` | `await ctx.step('id', fn)` |
21+
| Agent invocation | `yield* agents.writer(input)` (via `bindAgents`) | n/a — must be ported |
22+
| Engine consumes | `StepDescriptor` union (`agent`, `step`, `signal`, `approval`, `now`, `uuid`, `nested-workflow`, `patched`) | Method calls on `ctx` |
23+
| Pause mechanism | Generator yields pause descriptor → engine writes RunState + closes stream | Primitive throws `WorkflowPaused` sentinel → engine writes RunState + closes stream |
24+
25+
The two engines drive workflows fundamentally differently. They can coexist as two libraries, but they can't share the runtime path.
26+
27+
## Three integration options
28+
29+
### A. Two engines coexist
30+
31+
Leave `ai-orchestration` on its own generator engine; ship `workflow-core` separately. Maybe share a few types (`SchemaInput`, `InferSchema`).
32+
33+
- **Pro:** Zero refactor cost for ai-orchestration. PR 542 lands as-is.
34+
- **Con:** Two engines means two truths for the same concepts (replay rules, log shape, idempotency, run-state schema, version routing). Bug fixes diverge. Storage adapters have to be written twice. Supply-chain surface doubled.
35+
- **Verdict:** Don't pick this. The whole point of extracting `workflow-core` was to consolidate.
36+
37+
### B. Refactor ai-orchestration onto workflow-core (RECOMMENDED)
38+
39+
`ai-orchestration`'s engine code goes away. `defineAgent` and `defineOrchestrator` survive as AI-flavored sugar that produce `workflow-core` workflow definitions internally.
40+
41+
- **Pro:** One engine, one truth. Storage adapters and devtools written once. AI layer is genuinely a layer.
42+
- **Con:** Real refactor of ai-orchestration — most of the engine + primitives files get deleted. PR 542's reviewers see a smaller, AI-focused diff.
43+
- **Verdict:** Take this path.
44+
45+
### C. Ship both engine flavors in workflow-core
46+
47+
`workflow-core` exposes both a closure API (current) and a generator API (ported from PR 542). `ai-orchestration` picks the generator flavor.
48+
49+
- **Pro:** ai-orchestration's existing API is preserved verbatim.
50+
- **Con:** workflow-core becomes the union of two engine paradigms. Doubles surface area, doubles test matrix, doubles bug-fix work. Users have to pick a flavor. Migration story is muddier than just adopting closures.
51+
- **Verdict:** Don't pick this.
52+
53+
## What can be reliably removed from PR 542
54+
55+
Assuming Path B. These files have a direct equivalent in `@tanstack/workflow-core@0.0.1` and become re-exports or just deletions.
56+
57+
### Delete entirely (18 files)
58+
59+
| File | Replaced by |
60+
| ----------------------------------- | ----------------------------------------------------------------------- |
61+
| `src/engine/run-workflow.ts` | `runWorkflow` from `@tanstack/workflow-core` |
62+
| `src/engine/fingerprint.ts` | `workflow-core` (deprecated; explicit versioning preferred) |
63+
| `src/engine/state-diff.ts` | `workflow-core` (identical) |
64+
| `src/engine/emit-events.ts` | `workflow-core`'s `WorkflowEvent` shape |
65+
| `src/run-store/in-memory.ts` | `inMemoryRunStore` from `workflow-core` |
66+
| `src/server/parse-request.ts` | `parseWorkflowRequest` from `workflow-core` |
67+
| `src/server/index.ts` | re-export |
68+
| `src/registry/select-version.ts` | `selectWorkflowVersion` / `createWorkflowRegistry` from `workflow-core` |
69+
| `src/result.ts` | `succeed` / `fail` from `workflow-core` |
70+
| `src/define/define-workflow.ts` | `createWorkflow` from `workflow-core` |
71+
| `src/primitives/step.ts` | `ctx.step` |
72+
| `src/primitives/sleep.ts` | `ctx.sleep` / `ctx.sleepUntil` |
73+
| `src/primitives/wait-for-signal.ts` | `ctx.waitForEvent` |
74+
| `src/primitives/approve.ts` | `ctx.approve` |
75+
| `src/primitives/now.ts` | `ctx.now` |
76+
| `src/primitives/uuid.ts` | `ctx.uuid` |
77+
| `src/primitives/retry.ts` | `retry` from `workflow-core` (free function) |
78+
| `src/primitives/patched.ts` | Drop — replaced by `previousVersions` routing |
79+
| `src/primitives/bind-agents.ts` | Drop — agents become plain objects with `.invoke(...)` |
80+
81+
Plus most tests under `tests/` — `workflow-core` already covers the engine, durability, retry, timeout, idempotency, CAS, signals, primitives, in-memory-store, registry, parse-request, state-diff. The agent-specific ones (smoke uses agents heavily, durability, attach, publisher) get rewritten against the new shape.
82+
83+
### Keep + refactor (5-7 files)
84+
85+
| File | What changes |
86+
| ----------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
87+
| `src/define/define-agent.ts` | Agent shape changes from `run: → AgentRunResult` (returned to the generator engine) to `invoke(input, { emit, signal }): Promise<T> \| AsyncIterable<StreamChunk> \| { stream, output }` (called directly from a step body). Three-shape detection stays; just moves call site. |
88+
| `src/engine/invoke-agent.ts` | Keep as a pure helper. Called from inside a `ctx.step(...)` body in user code (or from the `withAgents` middleware). The three-shape detection + stream filtering + output parsing logic is genuinely AI-specific and not in workflow-core. |
89+
| `src/define/define-orchestrator.ts` | Rewritten as a thin wrapper around `createWorkflow().handler(async (ctx) => { /* router loop */ })`. The router itself stays as a function returning `RouterDecision`; instead of `yield*`-ing it inside a generator, it's called as `await router({...})` inside the closure. |
90+
| `src/define/define-router.ts` | Mostly intact — types-only helper. The router signature drops the `StepGenerator` return wrapping; it returns `Promise<RouterDecision>`. |
91+
| `src/types.ts` | Strip everything engine-shaped (StepDescriptor, StepGenerator, RunState, RunStore, etc. — those re-export from `workflow-core`). Keep AI-only types: `AgentDefinition`, `AgentRunArgs`, `AgentRunResult`, `AgentMap`, `BoundAgents`, `SchemaInput`-related helpers if AI uses them directly. |
92+
| `src/index.ts` | Public surface narrows to: `defineAgent`, `defineOrchestrator`, `defineRouter`, `invokeAgent`, `SchemaValidationError`, agent-related types. Re-exports from `workflow-core` for convenience. |
93+
| `src/middleware/with-agents.ts` (new, optional) | Provides a typed `ctx.agent(stepId, agentDef, input)` primitive via `createMiddleware` from `workflow-core`. Wraps `invokeAgent` + `ctx.step`. Pure ergonomic sugar. |
94+
95+
### Add
96+
97+
- `dependencies: { "@tanstack/workflow-core": "^0.0.1" }` to `package.json`.
98+
- Keep `peerDependencies: { "@tanstack/ai": "workspace:*" }` for the StreamChunk + chat APIs.
99+
100+
### Net result
101+
102+
```
103+
Before: 25 src files + 16 tests
104+
After: ~6 src files + ~6 AI-specific tests
105+
```
106+
107+
## Gaps in workflow-core (if any) for hosting the agent layer
108+
109+
Walked through Alem's article + orchestrator demos and Kyle's expense + AI-agent + durable-agent examples. **Zero engine changes are strictly required.** Three observations worth noting:
110+
111+
**1. Streaming side-effects from inside a step.** Agents return AG-UI chunks during execution. Today, a step body is just a callback that returns a value. To forward chunks: the step body closes over `ctx`, so `ctx.emit('chunk', chunk)` works inline. No engine change needed.
112+
113+
```ts
114+
const result = await ctx.step('writer', async () => {
115+
const { stream, output } = invokeAgent(writer, input, ctx.emit, ctx.signal)
116+
for await (const chunk of stream) ctx.emit(chunk.type, chunk as any)
117+
return output
118+
})
119+
```
120+
121+
Slight wart: looks like two `ctx.emit` plumbings (one passed to `invokeAgent`, one in the loop). `invokeAgent`'s `emit` argument is the agent's own emit hook for custom events; the stream-forwarding emit is separate. The `withAgents` middleware below hides this.
122+
123+
**2. Re-emit on replay.** When a step's result is cached, the chunks aren't re-emitted. A fresh attach sees `STEP_FINISHED` with the cached result, no inner deltas. This matches PR 542's behavior today ("per-token streaming history is not persisted; on attach mid-step the client sees STEP_STARTED with no prior tokens and then live tokens from the attach point onward"). No regression.
124+
125+
**3. Nested workflows.** PR 542's engine has a `nested-workflow` step descriptor. workflow-core does not. To "nest" workflows under the closure engine, call `runWorkflow({ workflow: child, ... })` from inside a step. The child consumes a fresh `runId` and is independent. Acceptable — orchestration doesn't need true nesting, and helpers wanting it can build it themselves in 20 lines.
126+
127+
## Optional polish: `withAgents` middleware
128+
129+
Adds `ctx.agent(stepId, agentDef, input)` to ctx. Pure sugar over `ctx.step` + `invokeAgent`. Lives in `ai-orchestration`, opt-in via middleware.
130+
131+
```ts
132+
// In ai-orchestration:
133+
export const withAgents = createMiddleware().server<{
134+
agent: <TInput, TOutput>(
135+
stepId: string,
136+
def: AgentDefinition<unknown, unknown, string>,
137+
input: TInput,
138+
) => Promise<TOutput>
139+
}>(async ({ ctx, next }) => {
140+
return next({
141+
context: {
142+
agent: (stepId, def, input) =>
143+
ctx.step(stepId, async () => {
144+
const { stream, output } = invokeAgent(
145+
def,
146+
input,
147+
ctx.emit,
148+
ctx.signal,
149+
)
150+
for await (const chunk of stream) {
151+
ctx.emit(chunk.type, chunk as unknown as Record<string, unknown>)
152+
}
153+
return output
154+
}),
155+
},
156+
})
157+
})
158+
```
159+
160+
User code:
161+
162+
```ts
163+
import { withAgents } from '@tanstack/ai-orchestration'
164+
import { createWorkflow } from '@tanstack/workflow-core'
165+
166+
const article = createWorkflow({
167+
id: 'article',
168+
input: z.object({ topic: z.string() }),
169+
})
170+
.middleware([withAgents])
171+
.handler(async (ctx) => {
172+
const draft = await ctx.agent('writer', writerAgent, {
173+
topic: ctx.input.topic,
174+
})
175+
const review = await ctx.agent('legal', legalAgent, { draft })
176+
if (review.verdict === 'block')
177+
return fail(`legal: ${review.findings.join('; ')}`)
178+
const decision = await ctx.approve({ title: 'Publish?' })
179+
if (!decision.approved) return fail('user denied')
180+
return succeed({ article: draft })
181+
})
182+
```
183+
184+
This reads identically in vibe to Alem's original generator-style code, just with `await ctx.agent(id, def, input)` instead of `yield* agents.name(input)`.
185+
186+
## Concrete before/after on one workflow
187+
188+
**Alem's article workflow today (PR 542 generator engine):**
189+
190+
```ts
191+
const articleWorkflow = defineWorkflow({
192+
name: 'article',
193+
input: ArticleInput,
194+
output: ArticleOutput,
195+
state: ArticleState,
196+
agents: { writer, legal, editor },
197+
run: async function* ({ input, state, agents }) {
198+
state.phase = 'drafting'
199+
const draft = yield* agents.writer({ topic: input.topic })
200+
state.draft = draft
201+
const review = yield* agents.legal({ draft })
202+
if (review.verdict === 'block')
203+
return fail(`legal: ${review.findings.join('; ')}`)
204+
const decision = yield* approve({ title: 'Publish?' })
205+
if (!decision.approved) return fail('user denied')
206+
return succeed({ article: draft })
207+
},
208+
})
209+
```
210+
211+
**Same workflow, ported (post-refactor):**
212+
213+
```ts
214+
const articleWorkflow = createWorkflow({
215+
id: 'article',
216+
input: ArticleInput,
217+
output: ArticleOutput,
218+
state: ArticleState,
219+
})
220+
.middleware([withAgents])
221+
.handler(async (ctx) => {
222+
ctx.state.phase = 'drafting'
223+
const draft = await ctx.agent('writer', writer, { topic: ctx.input.topic })
224+
ctx.state.draft = draft
225+
const review = await ctx.agent('legal', legal, { draft })
226+
if (review.verdict === 'block')
227+
return fail(`legal: ${review.findings.join('; ')}`)
228+
const decision = await ctx.approve({ title: 'Publish?' })
229+
if (!decision.approved) return fail('user denied')
230+
return succeed({ article: draft })
231+
})
232+
```
233+
234+
Two changes per call site: `yield*` → `await`, plus the agent reference now takes the step id as a first arg. Otherwise structurally identical. The `agents` declaration disappears (agents are just imports). The `state` schema flows through `.middleware([withAgents])` because middleware accumulation preserves the base ctx.
235+
236+
## Estimated refactor cost
237+
238+
Rough order:
239+
240+
| Task | Files | LoC delta | Time |
241+
| -------------------------------------------------------------------------------------- | ------------------ | --------- | ---------- |
242+
| Delete the 18 engine/primitive files | -18 src, -10 tests | -3500 | half a day |
243+
| Refactor `defineAgent` to closure-call shape | 1 src | small | 1 hour |
244+
| Refactor `defineOrchestrator` to closure-based router loop | 1 src | medium | 2 hours |
245+
| Update `defineRouter` types | 1 src | small | 30 min |
246+
| Strip engine types from `types.ts`, keep AI types | 1 src | medium | 1 hour |
247+
| Add `withAgents` middleware | 1 src + 1 test | small | 2 hours |
248+
| Update `index.ts` exports | 1 src | trivial | 15 min |
249+
| Add `@tanstack/workflow-core` dependency, build/test verify | configs | trivial | 30 min |
250+
| Rewrite AI-specific tests (smoke, durability, attach, publisher) against closure shape | ~4 tests | medium | 2-3 hours |
251+
252+
**Net: one focused day of work** to land PR 542 as an AI-only library on top of `workflow-core`.
253+
254+
## Open questions
255+
256+
- **AG-UI event mapping.** Workflow-core emits `WorkflowEvent` (`RUN_STARTED`, `STEP_FINISHED`, etc.). AG-UI clients (devtools, `@tanstack/ai-react`'s `useWorkflow` hook) expect the AG-UI `StreamChunk` shape. The two are intentional structural cousins. A small `toAgUiChunk(event)` adapter in `ai-client` is probably the cleanest seam.
257+
- **`@tanstack/ai-react`'s `useWorkflow` hook.** Currently expects the PR 542 engine's event stream. Will need to adapt to the workflow-core event shape (probably just the chunk-translation adapter above + `WorkflowEvent` → state reducer).
258+
- **Backwards-compat shim?** The PR 542 branch hasn't shipped, so there are no consumers to compat-break. Clean cut.
259+
- **Versioning.** `ai-orchestration` would land as its own pre-alpha (e.g., 0.0.1) once the refactor is done.
260+
261+
## Recommended sequence
262+
263+
1. **Land workflow-core 0.0.1** ✅ (done; on npm)
264+
2. **Refactor PR 542 onto workflow-core** following the file list above. Single focused day.
265+
3. **Cut `@tanstack/ai-orchestration@0.0.1`** alongside.
266+
4. **`@tanstack/ai-react`'s `useWorkflow` hook** updates to the workflow-core event shape — separate PR.
267+
5. **Devtools / attach UI** updates similarly.
268+
269+
Status: research only. Ready to execute when Tanner gives the word.

0 commit comments

Comments
 (0)