feat: publish personas, teams, and managed agents as relay events#939
Conversation
Ties the secrets-exclusion rule to the env_vars field removed in #939, making the doc-to-code mapping unambiguous. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…Body PersonaEventContent is the public struct serialized in kind:30175 events. PR #939 removed env_vars from it (secrets must not travel in plaintext events). This commit removes the re-addition from #945 and instead carries env_vars in PersonaEngramBody, which is NIP-44 encrypted inside the agent's mem/persona engram. Co-authored-by: Will Pfleger <wpfleger@block.xyz> Signed-off-by: Will Pfleger <wpfleger@block.xyz>
…Body PersonaEventContent is the public struct serialized in kind:30175 events. PR #939 removed env_vars from it (secrets must not travel in plaintext events). This commit removes the re-addition from #945 and instead carries env_vars in PersonaEngramBody, which is NIP-44 encrypted inside the agent's mem/persona engram. Co-authored-by: Will Pfleger <wpfleger@block.xyz> Signed-off-by: Will Pfleger <wpfleger@block.xyz>
9b8924a to
6c2e650
Compare
The Relay E2E job (new in #939) is the first to run these previously-skipped tests. Two asserted against `/channels/.../threads` and `/channels/.../messages` REST routes the relay never served (permanent 404); a third raced a live kind:44100 fan-out that a sibling subscription's drain silently discarded. Rewrite the thread read-backs against POST /query — the depth_limit + #e extension routes to get_thread_replies, the relay's real thread surface — and reorder the DM test to subscribe after create_dm so the persisted membership and discovery events are served deterministically from history. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Persona-created agents previously re-read the live persona catalog at every spawn, so a template edit silently changed running agents on restart. Snapshot the persona's system_prompt/model/provider/env_vars plus its content hash onto ManagedAgentRecord at create, and switch the spawn and provider-deploy call sites to read that snapshot. Restart reuses the record so it stays pinned; delete+respawn re-runs create and rewrites the snapshot. env_vars are pinned too — without that, persona credential edits would leak into a running agent on restart. The shared resolve_effective_prompt_model_provider body is left intact: its summary and ModelPicker callers intentionally read the live persona, so only the spawn/deploy call sites move to the snapshot. The Agents menu gets a drift indicator: build_managed_agent_summary compares the record's pinned source_version against the persona's current hash. A deleted persona is orphaned (never out-of-date, since there is nothing to respawn into). The summary now displays the pinned snapshot — what the agent actually runs — so the badge never contradicts the fields beside it. The mem/persona engram and fleet-update subsystem are removed: with the record authoritative, fleet-update became a self-referential write loop nothing read at spawn. Deletes fleet_update.rs, its launch and persona-save triggers, the engram-at-creation write, and the engram-only helpers in persona_events. The public persona catalog primitives (#939) are untouched — they remain the source the record snapshots from. Pre-existing agents are backfilled from the live persona once at launch, logging loudly if the linked persona is gone. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Persona-created agents previously re-read the live persona catalog at every spawn, so a template edit silently changed running agents on restart. Snapshot the persona's system_prompt/model/provider/env_vars plus its content hash onto ManagedAgentRecord at create, and switch the spawn and provider-deploy call sites to read that snapshot. Restart reuses the record so it stays pinned; delete+respawn re-runs create and rewrites the snapshot. env_vars are pinned too — without that, persona credential edits would leak into a running agent on restart. The shared resolve_effective_prompt_model_provider body is left intact: its summary and ModelPicker callers intentionally read the live persona, so only the spawn/deploy call sites move to the snapshot. The Agents menu gets a drift indicator: build_managed_agent_summary compares the record's pinned source_version against the persona's current hash. A deleted persona is orphaned (never out-of-date, since there is nothing to respawn into). The summary now displays the pinned snapshot — what the agent actually runs — so the badge never contradicts the fields beside it. The mem/persona engram and fleet-update subsystem are removed: with the record authoritative, fleet-update became a self-referential write loop nothing read at spawn. Deletes fleet_update.rs, its launch and persona-save triggers, the engram-at-creation write, and the engram-only helpers in persona_events. The public persona catalog primitives (#939) are untouched — they remain the source the record snapshots from. Pre-existing agents are backfilled from the live persona once at launch, logging loudly if the linked persona is gone. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Persona-created agents previously re-read the live persona catalog at every spawn, so a template edit silently changed running agents on restart. Snapshot the persona's system_prompt/model/provider/env_vars plus its content hash onto ManagedAgentRecord at create, and switch the spawn and provider-deploy call sites to read that snapshot. Restart reuses the record so it stays pinned; delete+respawn re-runs create and rewrites the snapshot. env_vars are pinned too — without that, persona credential edits would leak into a running agent on restart. The shared resolve_effective_prompt_model_provider body is left intact: its summary and ModelPicker callers intentionally read the live persona, so only the spawn/deploy call sites move to the snapshot. The Agents menu gets a drift indicator: build_managed_agent_summary compares the record's pinned source_version against the persona's current hash. A deleted persona is orphaned (never out-of-date, since there is nothing to respawn into). The summary now displays the pinned snapshot — what the agent actually runs — so the badge never contradicts the fields beside it. The mem/persona engram and fleet-update subsystem are removed: with the record authoritative, fleet-update became a self-referential write loop nothing read at spawn. Deletes fleet_update.rs, its launch and persona-save triggers, the engram-at-creation write, and the engram-only helpers in persona_events. The public persona catalog primitives (#939) are untouched — they remain the source the record snapshots from. Pre-existing agents are backfilled from the live persona once at launch, logging loudly if the linked persona is gone. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The Relay E2E job (new in #939) is the first to run these previously-skipped tests. Two asserted against `/channels/.../threads` and `/channels/.../messages` REST routes the relay never served (permanent 404); a third raced a live kind:44100 fan-out that a sibling subscription's drain silently discarded. Rewrite the thread read-backs against POST /query — the depth_limit + #e extension routes to get_thread_replies, the relay's real thread surface — and reorder the DM test to subscribe after create_dm so the persisted membership and discovery events are served deterministically from history. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
93a291e to
289bfc2
Compare
The Relay E2E job (new in #939) is the first to run these previously-skipped tests. Two asserted against `/channels/.../threads` and `/channels/.../messages` REST routes the relay never served (permanent 404); a third raced a live kind:44100 fan-out that a sibling subscription's drain silently discarded. Rewrite the thread read-backs against POST /query — the depth_limit + #e extension routes to get_thread_replies, the relay's real thread surface — and reorder the DM test to subscribe after create_dm so the persisted membership and discovery events are served deterministically from history. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
289bfc2 to
2fb370f
Compare
Persona-created agents previously re-read the live persona catalog at every spawn, so a template edit silently changed running agents on restart. Snapshot the persona's system_prompt/model/provider/env_vars plus its content hash onto ManagedAgentRecord at create, and switch the spawn and provider-deploy call sites to read that snapshot. Restart reuses the record so it stays pinned; delete+respawn re-runs create and rewrites the snapshot. env_vars are pinned too — without that, persona credential edits would leak into a running agent on restart. The shared resolve_effective_prompt_model_provider body is left intact: its summary and ModelPicker callers intentionally read the live persona, so only the spawn/deploy call sites move to the snapshot. The Agents menu gets a drift indicator: build_managed_agent_summary compares the record's pinned source_version against the persona's current hash. A deleted persona is orphaned (never out-of-date, since there is nothing to respawn into). The summary now displays the pinned snapshot — what the agent actually runs — so the badge never contradicts the fields beside it. The mem/persona engram and fleet-update subsystem are removed: with the record authoritative, fleet-update became a self-referential write loop nothing read at spawn. Deletes fleet_update.rs, its launch and persona-save triggers, the engram-at-creation write, and the engram-only helpers in persona_events. The public persona catalog primitives (#939) are untouched — they remain the source the record snapshots from. Pre-existing agents are backfilled from the live persona once at launch, logging loudly if the linked persona is gone. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Persona-created agents previously re-read the live persona catalog at every spawn, so a template edit silently changed running agents on restart. Snapshot the persona's system_prompt/model/provider/env_vars plus its content hash onto ManagedAgentRecord at create, and switch the spawn and provider-deploy call sites to read that snapshot. Restart reuses the record so it stays pinned; delete+respawn re-runs create and rewrites the snapshot. env_vars are pinned too — without that, persona credential edits would leak into a running agent on restart. The shared resolve_effective_prompt_model_provider body is left intact: its summary and ModelPicker callers intentionally read the live persona, so only the spawn/deploy call sites move to the snapshot. The Agents menu gets a drift indicator: build_managed_agent_summary compares the record's pinned source_version against the persona's current hash. A deleted persona is orphaned (never out-of-date, since there is nothing to respawn into). The summary now displays the pinned snapshot — what the agent actually runs — so the badge never contradicts the fields beside it. The mem/persona engram and fleet-update subsystem are removed: with the record authoritative, fleet-update became a self-referential write loop nothing read at spawn. Deletes fleet_update.rs, its launch and persona-save triggers, the engram-at-creation write, and the engram-only helpers in persona_events. The public persona catalog primitives (#939) are untouched — they remain the source the record snapshots from. Pre-existing agents are backfilled from the live persona once at launch, logging loudly if the linked persona is gone. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The Relay E2E job (new in #939) is the first to run these previously-skipped tests. Two asserted against `/channels/.../threads` and `/channels/.../messages` REST routes the relay never served (permanent 404); a third raced a live kind:44100 fan-out that a sibling subscription's drain silently discarded. Rewrite the thread read-backs against POST /query — the depth_limit + #e extension routes to get_thread_replies, the relay's real thread surface — and reorder the DM test to subscribe after create_dm so the persisted membership and discovery events are served deterministically from history. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
e6a44d7 to
7c98207
Compare
Persona-created agents previously re-read the live persona catalog at every spawn, so a template edit silently changed running agents on restart. Snapshot the persona's system_prompt/model/provider/env_vars plus its content hash onto ManagedAgentRecord at create, and switch the spawn and provider-deploy call sites to read that snapshot. Restart reuses the record so it stays pinned; delete+respawn re-runs create and rewrites the snapshot. env_vars are pinned too — without that, persona credential edits would leak into a running agent on restart. The shared resolve_effective_prompt_model_provider body is left intact: its summary and ModelPicker callers intentionally read the live persona, so only the spawn/deploy call sites move to the snapshot. The Agents menu gets a drift indicator: build_managed_agent_summary compares the record's pinned source_version against the persona's current hash. A deleted persona is orphaned (never out-of-date, since there is nothing to respawn into). The summary now displays the pinned snapshot — what the agent actually runs — so the badge never contradicts the fields beside it. The mem/persona engram and fleet-update subsystem are removed: with the record authoritative, fleet-update became a self-referential write loop nothing read at spawn. Deletes fleet_update.rs, its launch and persona-save triggers, the engram-at-creation write, and the engram-only helpers in persona_events. The public persona catalog primitives (#939) are untouched — they remain the source the record snapshots from. Pre-existing agents are backfilled from the live persona once at launch, logging loudly if the linked persona is gone. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Persona-created agents previously re-read the live persona catalog at every spawn, so a template edit silently changed running agents on restart. Snapshot the persona's system_prompt/model/provider/env_vars plus its content hash onto ManagedAgentRecord at create, and switch the spawn and provider-deploy call sites to read that snapshot. Restart reuses the record so it stays pinned; delete+respawn re-runs create and rewrites the snapshot. env_vars are pinned too — without that, persona credential edits would leak into a running agent on restart. The shared resolve_effective_prompt_model_provider body is left intact: its summary and ModelPicker callers intentionally read the live persona, so only the spawn/deploy call sites move to the snapshot. The Agents menu gets a drift indicator: build_managed_agent_summary compares the record's pinned source_version against the persona's current hash. A deleted persona is orphaned (never out-of-date, since there is nothing to respawn into). The summary now displays the pinned snapshot — what the agent actually runs — so the badge never contradicts the fields beside it. The mem/persona engram and fleet-update subsystem are removed: with the record authoritative, fleet-update became a self-referential write loop nothing read at spawn. Deletes fleet_update.rs, its launch and persona-save triggers, the engram-at-creation write, and the engram-only helpers in persona_events. The public persona catalog primitives (#939) are untouched — they remain the source the record snapshots from. Pre-existing agents are backfilled from the live persona once at launch, logging loudly if the linked persona is gone. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Persona create, edit, and delete now reach the relay so personas persist across reboots and reflect in other Buzz clients. #939 built the local retention primitives but left them inert; this wires every persona writer to a single publish path. pending_sync is the seam: each writer (UI create/edit, delete tombstone, launch reconcile) retains a signed event locally with pending_sync = 1 while holding the managed-agents store lock, and one background flush loop is the sole relay publisher. Delete purges the persona's retention row inside the lock-held command body (closing the same-second resurrect race) and enqueues a NIP-09 a-tag-only kind:5 tombstone for the flush loop to publish out of band. mark_synced is compare-and-clear on created_at+content so an edit landing mid-flush is never falsely cleared. Ownership is intrinsic to the signing key, so the only skip anywhere is is_builtin. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Teams become authoritative on the same kind-agnostic pending_sync/flush spine as personas (#939). A new kind:30176 NIP-33 parameterized-replaceable event projects the public team fields (name, description, persona_ids); the d_tag is the team id. create_team always publishes (created teams are never built-in), update_team skips built-in teams, and delete_team enqueues an a-tag-only kind:5 tombstone after the cascade delete — which itself rejects built-ins, so reaching the tombstone guarantees the team was owner-published. TeamEventContent is an explicit opt-IN projection so a future TeamRecord field is published only when deliberately added; local install fields (source_dir, is_symlink, is_builtin, timestamps) stay on-device. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Managed agents become authoritative on the same kind-agnostic pending_sync/ flush spine as personas (#939) and teams. A new kind:30177 NIP-33 parameterized-replaceable event is owner-signed with the agent's pubkey as d_tag, so the coordinate is 30177:<owner>:<agent_pubkey>. ManagedAgentEventContent is an explicit opt-IN allowlist, never record-minus-denylist: the flush loop is a blind pre-signed pipe, so this projection is the only guard against leaking secrets to a world-readable relay. It deliberately omits private_key_nsec, auth_tag, env_vars, the provider backend blob, and every runtime/install field. A future record field defaults to NOT published. Because the retention upsert compares the serialized projection, an operational start/stop that mutates only excluded runtime fields produces an identical row and never re-enqueues a publish — start/stop is a no-op by construction. Wiring follows the #939 async discipline: retain runs after save inside the store lock with no MutexGuard held across any .await, and the delete tombstone runs only past the deployed-remote guard so a live remote deployment is never orphaned. Standalone agents (persona_id None) publish too — team membership does not gate publishing. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The desktop publishes team (30176) and managed-agent (30177) NIP-33 events on the #939 edit->publish spine, but the relay rejected both at the required_scope_for_kind catch-all, so the new event kinds could never round-trip. Register them in the same UsersWrite arm as KIND_PERSONA (owner-authored, user-scoped state, keyed by signing key) and mark both global-only so a stray h tag never channel-scopes them. The persona slug envelope is not extended: team ids and 64-hex agent pubkeys satisfy the generic parameterized-replaceable d_tag path (D_TAG_MAX_LEN only). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Mirrors the persona e2e harness (#939) for the two new NIP-AP kinds: publish-and-query by NIP-33 coordinate, newer-timestamp-wins replacement, and a-tag-only tombstone-to-zero. For managed agents, one case asserts the content round-tripped through the relay carries none of the opt-IN-excluded fields (private_key_nsec, auth_tag, env_vars, backend blob, runtime fields) — proving the full publish->relay->query pipe, not just the projection function the unit tests already cover. Team d_tags exercise the generic parameterized-replaceable d_tag path (D_TAG_MAX_LEN only), distinct from the persona slug envelope. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Pinky review findingsI found two CHANGE items in the consumer-side sync surface. The relay/storage/projection side looks careful overall, but another device does not reliably catch up to existing/deleted records. CHANGE 1: Fresh consumers never backfill existing persona/team/agent events
relayClient.subscribeLive(
{ kinds: PERSONA_SYNC_KINDS, authors: [pubkey], limit: 0 },
...
)The relay client sends that filter as-is for the initial Suggested fix: either subscribe with a replay-capable positive limit (and page if necessary), or do an explicit one-shot history fetch for Relevant code:
CHANGE 2: Deletes/tombstones are published but never consumed by desktop syncThe delete paths correctly enqueue kind:5 Suggested fix: include Relevant code:
Validation from my local review pass:
|
The Relay E2E job (new in #939) is the first to run these previously-skipped tests. Two asserted against `/channels/.../threads` and `/channels/.../messages` REST routes the relay never served (permanent 404); a third raced a live kind:44100 fan-out that a sibling subscription's drain silently discarded. Rewrite the thread read-backs against POST /query — the depth_limit + #e extension routes to get_thread_replies, the relay's real thread surface — and reorder the DM test to subscribe after create_dm so the persisted membership and discovery events are served deterministically from history. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
589435e to
633bc63
Compare
Squashed for rebase — original commits: - feat(desktop): add persona event kind with client publish/read/retain - fix(desktop): sign every migrated persona event and drop the sentinel file - fix(relay): validate kind:30175 persona d-tag slug grammar on ingest - fix(desktop): drop env_vars from public PersonaEventContent - docs: add NIP-AP spec for kind:30175 persona events - feat(ci): add relay E2E testing job and persona event tests
Missed during rebase: sprout_core→buzz_core_pkg imports in persona_events.rs and migration.rs, sprout-desktop→buzz-desktop in log messages, and cargo fmt on e2e_persona.rs.
cargo fmt requires buzz_core_pkg imports before nostr (alphabetical). Relay E2E git tests need git-credential-nostr built alongside the relay.
…ay behavior The relay now has a generic file upload path that accepts PDFs, random bytes, and unrecognised formats as application/octet-stream downloads. SVG with XML declaration is not detected by `infer` and also routes through the generic path. Updated four content validation tests from expecting rejection (400/415) to expecting acceptance (200). Additionally, three WebSocket imeta tests hit /api/events but the relay route is at /events (no /api prefix). Fixed the URL in all three. Co-authored-by: Will Pfleger <wpfleger@squareup.com> Signed-off-by: Will Pfleger <wpfleger@squareup.com>
infer detects XML-based SVG as text/xml (not image/svg+xml), which is not in the blocked list, so the relay accepts it through the generic file path with that MIME type. Co-authored-by: Will Pfleger <wpfleger96@gmail.com> Signed-off-by: Will Pfleger <wpfleger96@gmail.com>
Same class of issues as e2e_media_extended — tests assumed image-only policy and wrong API path. The relay ignores Content-Type headers (uses magic bytes) and the route is /events not /api/events. Co-authored-by: Will Pfleger <wpfleger96@gmail.com> Signed-off-by: Will Pfleger <wpfleger96@gmail.com>
The live_split_model_completes test is a manual runbook test requiring multiple serve nodes. Replace panic!() with println+return so it skips gracefully when CI runs --ignored tests. Co-authored-by: Will Pfleger <wpfleger96@gmail.com> Signed-off-by: Will Pfleger <wpfleger96@gmail.com>
The Relay E2E job (new in #939) is the first to run these previously-skipped tests. Two asserted against `/channels/.../threads` and `/channels/.../messages` REST routes the relay never served (permanent 404); a third raced a live kind:44100 fan-out that a sibling subscription's drain silently discarded. Rewrite the thread read-backs against POST /query — the depth_limit + #e extension routes to get_thread_replies, the relay's real thread surface — and reorder the DM test to subscribe after create_dm so the persisted membership and discovery events are served deterministically from history. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
test_nip10_thread_reply_not_in_top_level asserted only that a reply threads under its root — a correlate, not the relay's actual top-level rule. get_channel_messages_top_level surfaces a depth-1 reply iff broadcast = true, so a broadcast=1 depth-1 reply satisfied every old assertion yet IS surfaced at top level: the test greened by data accident. The relay exposes no top-level-queryable surface over POST /query (feed_types routes to feed queries that never read thread_metadata.depth/broadcast), so the rule is pinned via its two test-observable inputs — recorded depth and the broadcast tag — in both directions: a non-broadcast depth-1 reply is excluded, a broadcast=1 depth-1 reply is surfaced. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
cargo fmt --all --check failed Rust Lint on the assertions added in the prior commit (lines 925, 938). Whitespace/reflow only; no assertion or logic change. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The relay's clock-skew guard rejects events with created_at too far from server time. Replace hardcoded Nov 2023 timestamps (1_700_000_000) with Timestamp::now()-relative values that stay within the skew window while preserving the older-vs-newer replacement semantics. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…2e_relay tests
Tests assumed HTTP REST endpoints (/api/events, /api/users/{pubkey}/profile,
/api/users/me/channel-add-policy) that the relay does not serve. The relay
router only exposes /events, /query, /count, and a few other paths.
Three fix patterns applied:
- POST /api/events → POST /events (the relay's actual HTTP bridge)
- GET /api/users/{pubkey}/profile → POST /query with kind:0 filter
- PUT /api/users/me/channel-add-policy → submit kind:10100 event via POST /events
Also fixed self-add test that hit nostr crate's default p-tag stripping
(EventBuilder removes p tags matching the signer unless allow_self_tagging
is called), and added a 1s sleep in kind0_nip05_sync to ensure the
replacement event gets a strictly newer created_at timestamp.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The relay-e2e job (introduced on this branch, absent on main) ran the full `--test '*'` glob across all buzz-test-client e2e suites. That swept in three suites targeting a REST `/api/*` surface that no longer exists in the relay binary — the surface was migrated to the Nostr HTTP bridge and torn out, leaving e2e_rest_api (37), e2e_tokens (19), and e2e_workflows (7) as 63 tests that 404 against a route the relay does not mount. This job was added to exercise the persona/interop work, so scope it to the two suites it covers: e2e_persona and e2e_nostr_interop. Reimplementing the REST surface is separate, non-gating work and should not block this merge base from going green. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co>
The outbound half already retains secrets-stripped persona/team/agent projections and flushes them to the relay, but nothing read events back, so a second device never inherited the first device's edits. Add the inbound consumer (C1) as a sync channel that patches projected fields INTO personas.json — retention is never an authoritative read source, so load_personas and every agent's UUID-based resolution and provider keys stay intact. Inbound events match the local record whose persona_d_tag equals the event d-tag (one derivation, both directions), so an in-app persona matches its existing UUID row instead of minting a duplicate; the retention store resolves equal-second collisions (preserve pending) versus strictly-newer events (inbound wins). The frontend subscribes via the existing relayClientSession live path; a pubkey-keyed effect re-establishes on identity change and replayLiveSubscriptions re-fires on reconnect. Fix two boot-order gaps (C2): run identity-independent migrations before identity resolution via migration::run_boot_migrations, and extend the boot content-hash reconcile to team metadata (kind 30176) alongside personas. Delete the parked load_from_retention read primitive. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…t boot migrate_personas_in_dir reads personas.json fresh and reconciles it into the retention store, but it ran before sync_team_personas, which writes team-dir edits into personas.json. So a team-dir persona edit was not reconciled to retention — and thus did not reach the relay — until the next launch. Run sync before the event migrations so the writer precedes the reader and a team-dir edit propagates same-launch. The coupling is single-directional (sync touches only the JSON stores, migrate only retention.db), so reordering is safe. Extracts the post-identity sync + event-reconcile sequence into migration::run_event_sync. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
reconcile_inbound_persona_event previously no-oped on kind:30176 (team) and kind:30177 (managed-agent), so those projections never propagated to a second device — only 30175 personas did. Implement both inbound paths. The managed-agent path is the secret-bearing one: it parses into the ManagedAgentEventContent projection (10 public fields), NOT a full ManagedAgentRecord. The projection type cannot represent private_key_nsec, auth_tag, env_vars, backend, or agent_command/agent_command_override, so a hostile inbound event carrying those keys has them dropped at deserialization — the structural guard, not a post-parse filter. The apply overwrites only the projected fields and preserves every device-local secret and runtime field. No match is a no-op: an agent has no secret key to run with unless it already exists locally. The team path is symmetric on the secret discipline but inserts on no match, since a team (like a persona) is a secretless definition another device may legitimately learn from the relay. Local install fields (source_dir, is_symlink, version) are preserved. Per-kind dispatch happens before any d-tag comparison, so a d-tag shared across kinds can never cross-link a team to a persona or agent. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Address seven review findings on the persona/team/managed-agent NIP-AP sync:
F1 — monotonic created_at. The three event builders defaulted created_at to
now; the local retention upsert replaces on >=, so a same-second second edit
was kept locally while the relay's NIP-33 lowest-id tiebreak could keep the
older event. Each retain path now reads the retained head and bumps to
max(now, head+1) per NIP-AP step 3.
F2 — consume kind:5 tombstones. Tombstones were published but never consumed,
so a delete never reached a second device. The reconcile command now handles
kind:5 (parse the a-tag coordinate, remove the local record per-kind under the
same store lock + retention spine as the apply path), and the FE subscription
covers KIND_DELETION. Retained tombstones are keyed by the full target
coordinate so cross-kind d-tag collisions occupy distinct rows.
F3 — fresh-start backfill. The live subscription used limit:0, and
reconnect-replay's since-cursor is undefined until a live event arrives, so a
device coming online after another published got zero history. The sync now
does a one-shot history fetch (including the deletion kind) before going live.
F4 — cascade persona tombstones. Deleting a directory-backed team removed its
sourced personas locally but left their kind:30175 heads live on the relay;
the cascade now returns the removed personas' d-tags and tombstones each.
F5 — d-tag normalization. Pack slugs are [a-zA-Z0-9_-]+, but the relay grammar
(matching the spec) requires ^[a-z0-9][a-z0-9_-]{0,63}$, so slugs like
CodeReviewer were rejected and stuck pending. persona_d_tag now normalizes at
the publish boundary (the spec's own write step), not by loosening the relay.
F6 — drop env_vars from the KIND_PERSONA doc-comment (spec forbids it; code was
already correct).
F7 — align PersonaEventContent field order to the NIP-AP reference vectors
(display_name, system_prompt, avatar_url, ...) so the serialized content bytes
and resulting NIP-01 event id match across implementations. Pinned by a
byte-exact vector test.
The secret-guard projection types (PersonaEventContent/TeamEventContent/
ManagedAgentEventContent) keep their exact field sets — F7 only reorders
PersonaEventContent's existing fields, no field added or removed.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
633bc63 to
71c0bcc
Compare
Summary
Establishes the publish/emit foundation for personas, teams, and managed agents as Nostr relay events, and wires every create/edit/delete in Buzz desktop to publish through it.
Three new NIP-33 parameterized-replaceable kinds carry the definitions, keyed on the owner's signing-key coordinate
(kind, pubkey, d_tag):301753017630177The behavior is a one-way publish mirror: a local change is written to the data-directory JSON (still the authoritative source of truth and read path) and also published to the relay as a derived, secrets-excluded projection. Nothing reads these events back yet —
load_personas/load_managed_agentsstill read JSON exclusively, andload_from_retentionis a parked primitive with no callers. This PR is the write half of a future sync system; it adds no user-visible behavior change on its own. The user-visible payoff (multi-device sync, sharing/discovery, headless reconstruction) arrives with a later consumer-side phase that subscribes and reads events back.Changes
Relay side
crates/buzz-core/src/kind.rs: RegisterKIND_PERSONA = 30175,KIND_TEAM = 30176,KIND_MANAGED_AGENT = 30177as NIP-33 parameterized replaceable, keyed by(pubkey, kind, d_tag)whered_tagis the plaintext slug. Compile-time assertions verify range membership.crates/buzz-relay/src/handlers/ingest.rs: Allowlist all three kinds underScope::UsersWrite, marked global-only (never channel-scoped). Owner-coordinate ownership is enforced by construction — the author-pubkey match check rejects events whose author is not the authenticated identity, and addressable events keyed on(kind, pubkey, d_tag)cannot collide across owners. Tests verify scope and global-only classification for each kind.Desktop client side — event modules
managed_agents/persona_events.rs(new): SerializePersonaRecord↔ kind:30175 event; the kind-genericflush_pending_eventspublish loop; publish/fetch helpers via the relay HTTP API.managed_agents/team_events.rs(new): Serialize team records ↔ kind:30176 event.managed_agents/agent_events.rs(new): SerializeManagedAgentRecord↔ kind:30177 event.ManagedAgentEventContentis a named-field-only allowlist struct (no#[serde(flatten)]/ catch-all) —private_key_nsec,auth_tag,env_vars, the providerbackendblob, and all runtime fields are absent by construction.managed_agents/retention.rs(new): SQLite retention store withINSERT OR REPLACEon(kind, pubkey, d_tag)for NIP-33 latest-wins semantics, plus a pending-sync queue for deferred relay publish. WAL +busy_timeouthandle the flush-vs-command write race;mark_syncedis compare-and-clear oncreated_at+contentso a newer edit landing mid-flush stays pending and republishes on the next pass.Desktop client side — wiring
commands/personas.rs,commands/teams.rs,commands/agents.rs: Each create/update/delete retains a pending event and tombstones on delete. Delete paths purge the pending row before writing the kind:5 tombstone (no-resurrect ordering); built-ins and team-referenced records are rejected before the d_tag is captured. The agent retain helper short-circuits on projection equality so runtime churn (pid/error/timestamp) produces no republish; the team and persona helpers intentionally omit the short-circuit (their republishes fire only on genuine user edits) and document the omission.lib.rs: A 30s spawn loop drives the single kind-genericflush_pending_events, re-fetching state per tick. NoMutexGuardis held across an.awaitanywhere in the retain/tombstone/flush paths — every connection and keys-lock is scoped and dropped before publish.migration.rs:migrate_personas_to_eventsruns after identity resolution, signs every retained event with the owner's real keys (no placeholder events), idempotent via retention-row-exists check.Spawn config pinned to the agent record (originally #945)
system_prompt/model/provider/env_varsplus a content hash onto theManagedAgentRecord. Spawn and deploy read that pinned snapshot, never the live persona — a running agent stays on the config it was created with across restarts.#[serde(default)]new fields mean oldmanaged-agents.jsonfiles deserialize cleanly, no migration needed.Key design decisions
nsec,auth_tag,env_vars, provider keys, and runtime state are excluded by construction. A non-vacuous unit test feeds real secret values and asserts none survive serialization. This is what makes future sync/sharing safe.ManagedAgentRecordsnapshot is the source of truth for spawn/deploy, so template edits never silently mutate running agents.personas.jsonis not deprecated — the data-directory JSON files remain the source of truth and the sole read path. The relay is a derived mirror, not a replacement. Creating/editing agents stays local in the Buzz desktop UI (or by hand-editing the JSON).