fix: resolve 18 correctness, durability, and honesty gaps (#47)

* fix: resolve 18 correctness, durability, and honesty gaps across the SDK

Agent core:
- EventStream.Wait() now returns the run's terminal error; ErrorDelta and
  the stream close error are emitted from a single point and always agree
- sub-agent failures mark the parent tool result as errored, and child
  agents inherit the parent's StepRunner (namespaced steps) for durability
- provider fallback now also triggers on mid-stream errors that arrive
  before any forwarded content; later errors propagate without duplication
- ConfigContent.Model is honored via the new types.ModelSwitcher seam,
  implemented by all four provider adapters

Durability:
- every tree mutation (archive/restore/checkpoint/rewind/branch/compact/
  set-active) writes WAL ops in a single transaction
- new agent/store/filewal: production append-only JSONL WAL with fsync on
  commit and torn-tail repair; agent/store/walrecover heals a store from
  committed-unapplied transactions; RecoverAndLoadTree wires it at startup
- checkpoints round-trip: Store.ListCheckpoints (pgstore + memstore) feeds
  LoadTreeFromStore so rewind works after restore
- DBOS adapter docs now describe the real sequential durable execution

Providers & CLI:
- Generate(ctx, prompt) implemented on anthropic/openai/google adapters
  via shared types.GenerateText (previously ollama-only seam)
- MarkedTool implements RichTool: approval-wrapped rich results keep blocks
- PDFs map to native document/file parts in anthropic and openai adapters,
  so ContentSupport claims are now truthful for every provider
- new --embed-provider / SAIGE_EMBED_PROVIDER decouples the CLI embedding
  provider from the LLM provider

RAG:
- document writes are transactional (pgstore tx; sections/variants now
  actually persisted); replace keeps the prior doc on mid-ingest failure
- metadata filters push down into SQL, removing the limit*3 over-fetch
- partial retriever failure returns surviving hits + ErrPartialSearch;
  RRF k is configurable via WithFusionK
- WithGraph registers the graph retriever and deletes graph episodes on
  document delete/replace (knowledge gains DeleteEpisodes end to end)

Knowledge graph:
- GroupID is real tenant isolation: group-scoped entity uniqueness, dedup,
  relations, and search (schema migration included; empty group = legacy)
- search errors propagate (ErrPartialSearch for partial fan-out failures)
  and episode metadata persists to a JSONB column and rounds back

Eval:
- scorer/metric errors record per-case and are excluded from aggregates
  instead of aborting the suite or silently scoring zero

* fix: clear CI lint and vulnerability failures

- bump Go to 1.25.12 (GO-2026-5856 crypto/tls) and goldmark to v1.7.17
  (GO-2026-5320 XSS)
- extract embedVariants/writeDocument/enrichGraph from pipeline Ingest and
  retrieveAll/fuseHits from Search to bring cyclomatic complexity under the
  gocyclo threshold; no behavior change

* test: add DB-backed and end-to-end integration suites

- rag/pgstore: transactional document-tree persistence, atomic replace with
  mid-transaction rollback, metadata filter pushdown past the old limit*3
  window, JSONB filter semantics, vector ordering (7 tests)
- knowledge/pgstore: group isolation for entities/relations/search, episode
  metadata round-trip, DeleteEpisodes cascade, in-place migration upgrade
  from the historical unscoped schema (5 tests)
- integration/: opt-in E2E suite (Ollama/Postgres/DBOS-gated, hermetic
  skips) covering the full agent loop, handoffs, structured output, RAG+KG
  round trip with graph cleanup on delete, WAL crash recovery via
  RecoverAndLoadTree, DBOS durable runs and idempotent replay; includes
  docker-compose infra and Justfile recipes
- dbos: register RunInput/RunOutput/StepResult with gob — the serializer
  encodes them as interface values, so unregistered types fail at runtime
- tree.FromStore/LoadTreeFromStore accept tree.Option and RecoverAndLoadTree
  re-attaches the WAL, so recovered sessions keep write-ahead protection
- FallbackError.Error() now includes the underlying provider errors

* fix: resolve 10 verified findings from high-effort review

Migrations (data stranding):
- backfill kg_entity/kg_relation group_id from episode mentions where the
  group is unambiguous; cross-tenant-merged entities deliberately stay in
  the default group rather than leak between tenants
- backfill agent_branch/agent_checkpoint conversation_id from each row's
  root node via recursive CTE; orphans stay in the legacy namespace

Agent core:
- fallback relay: UsageDelta (emitted at message_start before content) no
  longer latches the no-fallback gate, and every relay return path drains
  an abandoned source stream so provider goroutines can't leak
- RunDurable uses stream.Wait() (authoritative close error) instead of a
  racy in-band ErrorDelta, and an empty response under a dead context is
  reported as ErrStreamCanceled, so cancelled runs can't masquerade as
  success
- ConfigContent.Model now propagates through fallback and retry wrappers
  (both implement ModelSwitcher); an unswitchable provider logs a warning
  instead of silently ignoring the requested model
- new Agent.Checkpoint persists checkpoints through the Store so they
  round-trip without WAL recovery; Tree.Checkpoint docs state the contract

Durability:
- filewal rolls back partial writes by truncating to the pre-write offset
  (self-disabling if rollback fails) so mid-log corruption is impossible
- filewal.Compact atomically rewrites the log keeping only unapplied
  transactions; RecoverWAL compacts after a successful pass, bounding
  growth to one session

RAG:
- Pipeline.Delete removes the store document before graph episodes, so a
  failed delete can no longer destroy graph facts for a live document

Author: urmzd

Justfile

@@ -60,6 +60,23 @@ validate:
 fuzz PACKAGE FUNC DURATION="30s":
     go test -fuzz={{FUNC}} -fuzztime={{DURATION}} {{PACKAGE}}
 
+# docker compose plugin or standalone docker-compose, whichever is installed
+compose := `docker compose version >/dev/null 2>&1 && echo "docker compose" || echo "docker-compose"`
+
+# Start local integration infra (pgvector Postgres on :5433)
+integration-up:
+    {{compose}} -f integration/docker-compose.yml up -d --wait postgres
+
+# Stop integration infra and delete its data
+integration-down:
+    {{compose}} -f integration/docker-compose.yml down -v
+
+# Run end-to-end integration tests (Ollama + Postgres + DBOS); see integration/README.md
+test-integration:
+    SAIGE_TEST_OLLAMA_HOST="${SAIGE_TEST_OLLAMA_HOST:-http://localhost:11434}" \
+    SAIGE_TEST_POSTGRES_DSN="${SAIGE_TEST_POSTGRES_DSN:-postgres://postgres:test@localhost:5433/postgres?sslmode=disable}" \
+    go test ./integration/ -v -count=1 -timeout 30m
+
 # Build docker image
 docker-build:
     docker build -t saige .