🔒 Unify the `private` visibility predicate (3-predicate divergence) + durable ownership + namespace required-scope

[alphaonedev]

🔒 Unify the private visibility predicate (3-predicate divergence) + durable ownership + namespace required-scope

Severity: security-high (cross-tenant private-memory leak / ownership-bypass on the recall+search read paths).
Parent: v0.8.0 EPIC #1709 — MUST be 100% completed before multi-agent enforcement is enabled.
§2-property (§3 scope test): strengthens §2.2 (coherent) — scope=private must mean the same thing on every read path — and is a hard prerequisite for safe multi-agent §2.1 (endpoint-resident) operation. Cross-tenant isolation is a release-blocker independent of property mapping.

Provenance. 5-agent adversarial assessment (2026-06-17) run against the codegraph index at /home/fate_two/v07/v07-f5 (release/v0.8.0). Every anchor below was codegraph-confirmed; line numbers may drift — re-verify at HEAD before editing.

---

🧨 The defect (codegraph-confirmed)

There are three independent implementations of the scope=private visibility check, and two of them are wrong (namespace-keyed instead of owner-keyed):

#	Predicate	Location	private means	Status
1	SQL visibility_clause	src/storage/mod.rs:330 (arm at :340)	scope_idx='private' AND m.namespace = caller_ns	❌ namespace-keyed — leaks
2	Rust is_visible (HNSW branch)	src/storage/mod.rs:192 (arm at :215)	&mem.namespace == ns	❌ namespace-keyed — leaks
3	Rust is_visible_to_caller	src/visibility.rs:46 (owner check :69)	metadata.agent_id == caller (+ target_agent_id inbox carve-out)	✅ canonical (#951), owner-keyed

Predicate #3 is the documented canonical (#951) and matches the postgres list clause (metadata->>'agent_id' = $6, src/store/postgres.rs:10508). Predicates #1 and #2 — on the recall, search, and hybrid (HNSW + linear-SQL) read paths — key "private" on namespace membership, not ownership.

Leak scenario

Agent alice stores a scope=private memory in fortitude/X. Agent bob, also positioned in fortitude/X (as_agent="fortitude/X"), runs memory_recall/memory_search: the clause scope_idx='private' AND m.namespace='fortitude/X' evaluates TRUE → bob reads alice's private memory. Via memory_list (predicate #3) the same row is correctly hidden. Same tool surface, two definitions of "private". The divergence is untested (both is_visible and compute_visibility_prefixes flagged "no covering tests"; the only owner-exclusion test, caller_non_owner_excludes_cross_agent_private, exists for list only).

Compounding gaps (Agent-5 + Agent-4 findings)

• Currently inert / lockout trap. resolve_read_visibility_caller() (src/identity/mod.rs:238) reads only AI_MEMORY_AGENT_ID; unset → None → trust-all (all filtering off). So scope=private is a no-op on read today. Owner stamps are PID-suffixed (ai:claude-code@host:pid-N, src/identity/mod.rs:184), non-durable across restarts — the moment anyone sets AI_MEMORY_AGENT_ID to enable filtering, every pre-existing private row is owned by a dead PID id → operator self-lockout.
• No namespace default/required scope. CorePolicy/GovernancePolicy (src/models/namespace.rs:478) has no scope field; namespace_meta stores only (standard_id, parent_namespace). The GOVERNANCE_PRE_WRITE hook is refuse-only (takes &Memory, returns Result<()>; Decision{Allow,Refuse,Warn,Escalate} — no mutate arm, src/daemon_runtime.rs:3229). So "force/default scope" is NOT a config toggle and NOT hangable as a silent default.
• Curator bypass. Curator reads with CallerContext::for_admin → bypass_visibility=true (src/store/mod.rs:431); it reads private rows and can distill them into consolidations.

---

✅ Work breakdown (all four workstreams = 100% done)

Workstream A — Unify the private predicate (the bug) · security-high

• A0 Verify whether sqlite has an agent_id_idx generated column (postgres does; sqlite likely does not).
• A1 Migration (src/storage/migrations.rs + register src/storage/migration_meta.rs): add VIRTUAL GENERATED agent_id_idx + target_agent_id_idx columns (mirror scope_idx at :1364) + partial indexes. ALTER TABLE ADD COLUMN <virtual generated> does not rebuild the table — assert "no rebuild" so triggers survive (per the migration-rebuild-drops-triggers lesson); keep the trigger suite in the gate.
• A2 Rewrite visibility_clause private arm (:340) → scope_idx='private' AND (agent_id_idx = ?caller OR target_agent_id_idx = ?caller). Keep team/unit/org namespace-keyed; keep collective open. Add the ?caller placeholder.
• A3 Thread caller: Option<&str> through recall (:2929), search (:2372), hybrid-linear-SQL (:8960) and bind at all call-sites; source from resolve_read_visibility_caller() at the SAL/handler boundary. caller (agent_id) is distinct from as_agent (namespace) — both must be plumbed.
• A4 Fix Rust is_visible HNSW branch (:215): make Private owner-keyed (delegate to canonical is_visible_to_caller); keep team/unit/org local; thread caller.
• A5 Anti-re-drift: single matrix test asserting SQL visibility_clause, Rust is_visible, and is_visible_to_caller agree across (scope × owner × caller × namespace).
• A6 Regression tests mirroring caller_non_owner_excludes_cross_agent_private for recall / search / hybrid-HNSW / hybrid-linear (bob in fortitude/X must NOT retrieve alice's private row). Plus: no-scope insert excluded for non-owner (pins scope_idx default-private → visibility contract).
• A7 Postgres parity: confirm PG recall/search are owner-keyed (PG list already is); add bob/alice parity test in tests/store_parity_gaps.rs.

Workstream B — Durable ownership + safe enablement · security-medium

• B1 Stop PID-suffixed owner stamps for the operator principal (src/identity/mod.rs:152-218): stable host-scoped id or mandate AI_MEMORY_AGENT_ID; set stable value in ~/.claude.json env + curator unit.
• B2 One-time owner-rewrite/claim tool (new CLI, e.g. ai-memory reown --namespace <ns> --to <id>): rewrite metadata.agent_id on existing rows BEFORE enabling filtering; include a "claim unowned/legacy (empty agent_id)" path (empty_owner_blocks_named_caller).
• B3 Lockout guard: startup check warns/refuses if AI_MEMORY_AGENT_ID is set but rows are owned by a different/pid-suffixed id. Hard order: A merged → B2 run → flip env.

Workstream C — Namespace required-scope (refuse-only) · enhancement

• C1 Add scope (+ owner) to the governance pre-write payload (src/daemon_runtime.rs:3169 + PG twin src/store/postgres.rs:8001).
• C2 Add required_scope: Option<MemoryScope> to CorePolicy (src/models/namespace.rs:478) with #[serde(default)]; tolerant deserialize of the namespace-standard metadata.governance blob.
• C3 Evaluate: write scope (absent ⇒ private default) ≠ required ⇒ Decision::Refuse + GOVERNANCE_REFUSED. Accept absent/default-private writes. (Refuse-only; coercion is the deferred large variant below.)
• C4 Surface in set_namespace_standard handler (src/handlers/hook_subscribers.rs); SDK parity (python/ts namespace-standard body; scope already on CreateMemoryRequest).
• C5 Tests both backends: scope:shared into a required_scope=private ns ⇒ refused; scope:private/absent ⇒ allowed.
• C-alt (DEFER — large) true silent default/coerce: a pre-validate mutation stage in src/storage::insert + both adapters that fills absent scope from resolved namespace policy before persist. Only if transparent inheritance becomes a hard requirement.

Workstream D — Curator / leak channel · security-medium

• D1 Decide whether curator (bypass_visibility=true, src/curator/reflection_pass.rs:109) may read private namespaces; if not, exclude them from autonomy/consolidation passes or run non-bypass.
• D2 Curator-written consolidations default scope=private owner ai:curator (reflection_pass.rs:290): decide intended owner/scope and stamp explicitly (avoid trust-all distillation leak now; avoid operator-invisible reflections once filtering is on).

Cross-cutting

• Three-surface parity (CLI/MCP/HTTP) + SDK parity for any new param.
• No hardcoded literals — scope/owner strings via MemoryScope/META_KEY_* SSOTs; literal-gate green.
• Any table rebuild recreates all triggers + runs the trigger suite in the gate.

---

🚦 Definition of done (100%)

1. Op-0 posture decision recorded (single-operator trust-all vs enforced multi-agent). Enforced ⇒ A+B mandatory before flipping enforcement.
2. All three predicates agree (A5 matrix test green); A6/A7 cross-namespace leak tests green on both backends.
3. Owner stamps durable; reown tool ships; lockout guard active.
4. Namespace required_scope (refuse-only) ships + configurable + SDK parity.
5. Curator posture (D1/D2) decided and enforced.
6. CHANGELOG declares §2.2 with code anchors (§17 gate).

📎 Cross-refs

EPIC #1709 (Phase 1 / release-gate). #951 (canonical is_visible_to_caller). Provenance: 5-agent adversarial assessment 2026-06-17.

🤖 Generated with Claude Code

[alphaonedev]

✅ Workstream A (the leak) — A1–A6 landed; A7 next

Driven by the autonomous v0.8.0 #1709 loop. The cross-tenant scope=private leak is closed on all read paths (sqlite).

• A0 ✅ Probed: sqlite ALREADY had agent_id_idx (the issue assumed it likely didn't) — so A1 collapsed to just target_agent_id_idx.
• A1 ✅ target_agent_id_idx VIRTUAL GENERATED column (sqlite) / STORED (pg) + index, schema v66→v67 lockstep both adapters, literal-67 stamp; in-place ADD COLUMN, triggers survive. Commit d9caad48.
• A2 ✅ visibility_clause private arm → owner-keyed scope_idx='private' AND ?caller IS NOT NULL AND (agent_id_idx=?caller OR target_agent_id_idx=?caller). New caller_ph placeholder; all 5 call-sites bind-audited (caller@start+4; trailing source_uri placeholders renumbered at the 4 affected sites).
• A3 ✅ caller: Option<&str> (resolve_read_visibility_caller — agent_id, ≠ as_agent namespace) threaded through recall/search/hybrid + ~18 MCP/HTTP/CLI/SAL call-sites; memory_list unchanged.
• A4 ✅ Rust is_visible HNSW Private arm delegates to canonical is_visible_to_caller. NULL caller → no private rows (fail-closed).
• A5 ✅ anti-re-drift matrix test: SQL visibility_clause == Rust is_visible == is_visible_to_caller (private owner-keyed in all three).
• A6 ✅ 15 leak regressions — confirmed the leak FAILED pre-fix on all 4 paths (recall/search/hybrid-HNSW/hybrid-linear), now blocked; owner + inbox target_agent_id carve-out retrieve; collective open; team-subtree guard. Commit 15e0504f.

Gates: clippy pedantic --tests (both adapters), leak 15/15, matrix, storage::tests 261, recall 162 (incl source_uri), search 50, ck_trigger 3, audit, 3 script gates, qual_10. No schema change beyond A1; no tool-count change.

Next: A7 — postgres recall/search owner-keying parity (pg list already owner-keys; the adapter compiles with caller threaded) + a bob/alice pg parity test. Then Workstreams B (durable ownership + reown + lockout), C (namespace required_scope), D (curator posture).

🤖 v0.8.0 #1709 loop (Claude Opus 4.8).

[alphaonedev]

✅ Workstream A (the leak) — 100% COMPLETE, both adapters

A7 landed (commit 75348c2c). The security-high cross-tenant scope=private leak is fully closed.

• A0–A6 ✅ sqlite owner-keyed predicate unification + 15 leak tests + anti-re-drift matrix (d9caad48, 15e0504f).
• A7 ✅ postgres parity + a bypass regression caught & fixed: the A2 owner-keying had regressed the sqlite admin/curator bypass path — with bypass_visibility=true + as_agent=Some, admin saw ZERO private rows (pre-A2 they saw them), because the trust-all sentinel is keyed on the namespace prefix and the SAL passed as_agent as-is on bypass. Fix: drop as_agent on bypass ((None,None)) so the sentinel fires → trust-all, matching postgres's NULL-caller bypass. Postgres recall/search/recall_hybrid were already owner-keyed with the target_agent_id carve-out + NULL-caller bypass (no pg fix). Tests: sqlite_admin_bypass_visibility_a7_1720 (6 — admin bypass sees private on all paths incl. as_agent=Some; non-admin stays owner-keyed) FAILED 3/6 pre-fix → 6/6 post-fix; store_parity_gaps pg parity test (gated on AI_MEMORY_TEST_POSTGRES_URL).

Invariant now holds on BOTH adapters: non-admin = owner-keyed (agent_id or target_agent_id == caller, fail-closed on NULL); admin/curator bypass = trust-all (sees private).

Next: Workstream B (durable ownership — stop PID-suffixed owner stamps, ai-memory reown CLI, lockout guard) → C (namespace required_scope, refuse-only) → D (curator posture). #1720 closes at DoD 100%.

🤖 v0.8.0 #1709 loop (Claude Opus 4.8).

[alphaonedev]

Workstream B — COMPLETE ✅ (durable identity + safe-opt-in scaffolding)

Following Workstream A (cross-tenant private leak closed both adapters: d9caad4 / 15e0504 / 75348c2), Workstream B is now landed in full on release/v0.8.0:

• B1 — durable pid-free owner stamps (e7994684). identity::resolve_agent_id ladder steps 3 (MCP clientInfo) + 4 (host fallback) no longer embed the live pid/uuid:
  • ai:<client>@<host>:pid-<pid> → ai:<client>@<host>
  • host:<host>:pid-<pid>-<uuid8> → host:<host>
    The owner stamp is now stable across process restarts — the prerequisite for safe enforced-multi-agent opt-in (a pid-suffixed stamp changes every boot, orphaning the operator's own scope=private rows the instant AI_MEMORY_AGENT_ID is set). process_discriminator() is UNCHANGED (still backs only the ephemeral anonymous: principals + anonymous HTTP req-ids). Filtering is NOT flipped on. Op-0 posture recorded in the module doc-comment + CLAUDE.md + 6 docs/*.md (ladder/leak lines synced). Non-breaking: new rows get the stable id, old pid-suffixed rows keep theirs until reown.
• B2 — ai-memory reown (e6787d5d, prior). Rewrites metadata.agent_id ownership across a namespace before enablement (--dry-run / --claim-unowned, both adapters, single-key json_set/jsonb_set).
• B3 — boot owner-lockout guard (39a4c2c3). run_mcp_server runs a cheap indexed COUNT at boot: when AI_MEMORY_AGENT_ID is set but pre-existing private rows are owned by a different / pid-suffixed / unowned id, it emits a stderr WARN naming ai-memory reown — or, under AI_MEMORY_REQUIRE_OWNED_ROWS, hard-refuses boot. No-op under the single-operator trust-all default. Closes the silent self-lockout trap.

Hard order satisfied: A done → B2 (reown tool) in hand → B1 (durable stamps) → B3 (guard). Enablement flip remains the operator's call; the substrate now makes that flip safe.

Gates on every unit: cargo fmt + clippy --features sal-postgres --tests (-D warnings -D clippy::all -D clippy::pedantic) + targeted unit tests + qual_10 module-size ceiling + the 3 script gates + cargo audit, all green. B1: 321 identity unit tests + new durability assertions. B3: 6 new unit tests (3 probe + 3 guard).

Next: Workstream C (namespace required_scope, refuse-only) → Workstream D (curator consolidation owner/scope posture) → DoD.