Skip to content

feat: publish personas, teams, and managed agents as relay events#939

Merged
wpfleger96 merged 18 commits into
mainfrom
wpfleger/persona-events
Jun 24, 2026
Merged

feat: publish personas, teams, and managed agents as relay events#939
wpfleger96 merged 18 commits into
mainfrom
wpfleger/persona-events

Conversation

@wpfleger96

@wpfleger96 wpfleger96 commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Establishes the publish/emit foundation for personas, teams, and managed agents as Nostr relay events, and wires every create/edit/delete in Buzz desktop to publish through it.

Three new NIP-33 parameterized-replaceable kinds carry the definitions, keyed on the owner's signing-key coordinate (kind, pubkey, d_tag):

Kind Record
30175 persona
30176 team
30177 managed agent

The behavior is a one-way publish mirror: a local change is written to the data-directory JSON (still the authoritative source of truth and read path) and also published to the relay as a derived, secrets-excluded projection. Nothing reads these events back yet — load_personas / load_managed_agents still read JSON exclusively, and load_from_retention is a parked primitive with no callers. This PR is the write half of a future sync system; it adds no user-visible behavior change on its own. The user-visible payoff (multi-device sync, sharing/discovery, headless reconstruction) arrives with a later consumer-side phase that subscribes and reads events back.

Changes

Relay side

  • crates/buzz-core/src/kind.rs: Register KIND_PERSONA = 30175, KIND_TEAM = 30176, KIND_MANAGED_AGENT = 30177 as NIP-33 parameterized replaceable, keyed by (pubkey, kind, d_tag) where d_tag is the plaintext slug. Compile-time assertions verify range membership.
  • crates/buzz-relay/src/handlers/ingest.rs: Allowlist all three kinds under Scope::UsersWrite, marked global-only (never channel-scoped). Owner-coordinate ownership is enforced by construction — the author-pubkey match check rejects events whose author is not the authenticated identity, and addressable events keyed on (kind, pubkey, d_tag) cannot collide across owners. Tests verify scope and global-only classification for each kind.

Desktop client side — event modules

  • managed_agents/persona_events.rs (new): Serialize PersonaRecord ↔ kind:30175 event; the kind-generic flush_pending_events publish loop; publish/fetch helpers via the relay HTTP API.
  • managed_agents/team_events.rs (new): Serialize team records ↔ kind:30176 event.
  • managed_agents/agent_events.rs (new): Serialize ManagedAgentRecord ↔ kind:30177 event. ManagedAgentEventContent is a named-field-only allowlist struct (no #[serde(flatten)] / catch-all) — private_key_nsec, auth_tag, env_vars, the provider backend blob, and all runtime fields are absent by construction.
  • managed_agents/retention.rs (new): SQLite retention store with INSERT OR REPLACE on (kind, pubkey, d_tag) for NIP-33 latest-wins semantics, plus a pending-sync queue for deferred relay publish. WAL + busy_timeout handle the flush-vs-command write race; mark_synced is compare-and-clear on created_at+content so a newer edit landing mid-flush stays pending and republishes on the next pass.

Desktop client side — wiring

  • commands/personas.rs, commands/teams.rs, commands/agents.rs: Each create/update/delete retains a pending event and tombstones on delete. Delete paths purge the pending row before writing the kind:5 tombstone (no-resurrect ordering); built-ins and team-referenced records are rejected before the d_tag is captured. The agent retain helper short-circuits on projection equality so runtime churn (pid/error/timestamp) produces no republish; the team and persona helpers intentionally omit the short-circuit (their republishes fire only on genuine user edits) and document the omission.
  • lib.rs: A 30s spawn loop drives the single kind-generic flush_pending_events, re-fetching state per tick. No MutexGuard is held across an .await anywhere in the retain/tombstone/flush paths — every connection and keys-lock is scoped and dropped before publish.
  • migration.rs: migrate_personas_to_events runs after identity resolution, signs every retained event with the owner's real keys (no placeholder events), idempotent via retention-row-exists check.

Spawn config pinned to the agent record (originally #945)

  • At persona-agent create time, snapshot the persona's system_prompt/model/provider/env_vars plus a content hash onto the ManagedAgentRecord. Spawn and deploy read that pinned snapshot, never the live persona — a running agent stays on the config it was created with across restarts.
  • A persona edit reaches an agent only via delete+respawn, which rewrites the snapshot; an "Out of date"/"Orphaned" drift badge shows in the meantime.
  • Pre-existing records are backfilled once at launch; #[serde(default)] new fields mean old managed-agents.json files deserialize cleanly, no migration needed.

Key design decisions

  1. One-way publish mirror — local JSON is authoritative and disk-write happens before/independent of publish; the relay event is a derived projection. Worst case if publish misbehaves is "events nobody reads yet," never corrupted local config.
  2. Secrets never serialize — the published projection is an opt-in allowlist of the shareable shape; nsec, auth_tag, env_vars, provider keys, and runtime state are excluded by construction. A non-vacuous unit test feeds real secret values and asserts none survive serialization. This is what makes future sync/sharing safe.
  3. d-tag: plaintext slug — personas/teams/agents are designed for discovery/sharing, not privacy. HMAC blinding is the wrong threat model.
  4. Retention-first write strategy — write to local SQLite first (durable copy), then publish to relay opportunistically. Success on local write, not relay ack; relay-unreachable leaves the row pending for the next flush pass.
  5. Record is authoritative for spawn config — once an agent is created, its ManagedAgentRecord snapshot is the source of truth for spawn/deploy, so template edits never silently mutate running agents.
  6. personas.json is not deprecated — the data-directory JSON files remain the source of truth and the sole read path. The relay is a derived mirror, not a replacement. Creating/editing agents stays local in the Buzz desktop UI (or by hand-editing the JSON).

@wpfleger96 wpfleger96 requested a review from a team as a code owner June 10, 2026 04:19
@wpfleger96 wpfleger96 marked this pull request as draft June 10, 2026 13:59
wpfleger96 added a commit that referenced this pull request Jun 10, 2026
Ties the secrets-exclusion rule to the env_vars field removed in #939,
making the doc-to-code mapping unambiguous.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 11, 2026
…Body

PersonaEventContent is the public struct serialized in kind:30175 events.
PR #939 removed env_vars from it (secrets must not travel in plaintext
events). This commit removes the re-addition from #945 and instead
carries env_vars in PersonaEngramBody, which is NIP-44 encrypted inside
the agent's mem/persona engram.

Co-authored-by: Will Pfleger <wpfleger@block.xyz>
Signed-off-by: Will Pfleger <wpfleger@block.xyz>
wpfleger96 added a commit that referenced this pull request Jun 11, 2026
…Body

PersonaEventContent is the public struct serialized in kind:30175 events.
PR #939 removed env_vars from it (secrets must not travel in plaintext
events). This commit removes the re-addition from #945 and instead
carries env_vars in PersonaEngramBody, which is NIP-44 encrypted inside
the agent's mem/persona engram.

Co-authored-by: Will Pfleger <wpfleger@block.xyz>
Signed-off-by: Will Pfleger <wpfleger@block.xyz>
@wpfleger96 wpfleger96 force-pushed the wpfleger/persona-events branch 2 times, most recently from 9b8924a to 6c2e650 Compare June 12, 2026 18:45
wpfleger96 added a commit that referenced this pull request Jun 16, 2026
The Relay E2E job (new in #939) is the first to run these previously-skipped
tests. Two asserted against `/channels/.../threads` and `/channels/.../messages`
REST routes the relay never served (permanent 404); a third raced a live
kind:44100 fan-out that a sibling subscription's drain silently discarded.

Rewrite the thread read-backs against POST /query — the depth_limit + #e
extension routes to get_thread_replies, the relay's real thread surface — and
reorder the DM test to subscribe after create_dm so the persisted membership
and discovery events are served deterministically from history.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 17, 2026
Persona-created agents previously re-read the live persona catalog at
every spawn, so a template edit silently changed running agents on
restart. Snapshot the persona's system_prompt/model/provider/env_vars
plus its content hash onto ManagedAgentRecord at create, and switch the
spawn and provider-deploy call sites to read that snapshot. Restart
reuses the record so it stays pinned; delete+respawn re-runs create and
rewrites the snapshot. env_vars are pinned too — without that, persona
credential edits would leak into a running agent on restart.

The shared resolve_effective_prompt_model_provider body is left intact:
its summary and ModelPicker callers intentionally read the live persona,
so only the spawn/deploy call sites move to the snapshot.

The Agents menu gets a drift indicator: build_managed_agent_summary
compares the record's pinned source_version against the persona's
current hash. A deleted persona is orphaned (never out-of-date, since
there is nothing to respawn into). The summary now displays the pinned
snapshot — what the agent actually runs — so the badge never contradicts
the fields beside it.

The mem/persona engram and fleet-update subsystem are removed: with the
record authoritative, fleet-update became a self-referential write loop
nothing read at spawn. Deletes fleet_update.rs, its launch and
persona-save triggers, the engram-at-creation write, and the
engram-only helpers in persona_events. The public persona catalog
primitives (#939) are untouched — they remain the source the record
snapshots from. Pre-existing agents are backfilled from the live
persona once at launch, logging loudly if the linked persona is gone.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 17, 2026
Persona-created agents previously re-read the live persona catalog at
every spawn, so a template edit silently changed running agents on
restart. Snapshot the persona's system_prompt/model/provider/env_vars
plus its content hash onto ManagedAgentRecord at create, and switch the
spawn and provider-deploy call sites to read that snapshot. Restart
reuses the record so it stays pinned; delete+respawn re-runs create and
rewrites the snapshot. env_vars are pinned too — without that, persona
credential edits would leak into a running agent on restart.

The shared resolve_effective_prompt_model_provider body is left intact:
its summary and ModelPicker callers intentionally read the live persona,
so only the spawn/deploy call sites move to the snapshot.

The Agents menu gets a drift indicator: build_managed_agent_summary
compares the record's pinned source_version against the persona's
current hash. A deleted persona is orphaned (never out-of-date, since
there is nothing to respawn into). The summary now displays the pinned
snapshot — what the agent actually runs — so the badge never contradicts
the fields beside it.

The mem/persona engram and fleet-update subsystem are removed: with the
record authoritative, fleet-update became a self-referential write loop
nothing read at spawn. Deletes fleet_update.rs, its launch and
persona-save triggers, the engram-at-creation write, and the
engram-only helpers in persona_events. The public persona catalog
primitives (#939) are untouched — they remain the source the record
snapshots from. Pre-existing agents are backfilled from the live
persona once at launch, logging loudly if the linked persona is gone.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 17, 2026
Persona-created agents previously re-read the live persona catalog at
every spawn, so a template edit silently changed running agents on
restart. Snapshot the persona's system_prompt/model/provider/env_vars
plus its content hash onto ManagedAgentRecord at create, and switch the
spawn and provider-deploy call sites to read that snapshot. Restart
reuses the record so it stays pinned; delete+respawn re-runs create and
rewrites the snapshot. env_vars are pinned too — without that, persona
credential edits would leak into a running agent on restart.

The shared resolve_effective_prompt_model_provider body is left intact:
its summary and ModelPicker callers intentionally read the live persona,
so only the spawn/deploy call sites move to the snapshot.

The Agents menu gets a drift indicator: build_managed_agent_summary
compares the record's pinned source_version against the persona's
current hash. A deleted persona is orphaned (never out-of-date, since
there is nothing to respawn into). The summary now displays the pinned
snapshot — what the agent actually runs — so the badge never contradicts
the fields beside it.

The mem/persona engram and fleet-update subsystem are removed: with the
record authoritative, fleet-update became a self-referential write loop
nothing read at spawn. Deletes fleet_update.rs, its launch and
persona-save triggers, the engram-at-creation write, and the
engram-only helpers in persona_events. The public persona catalog
primitives (#939) are untouched — they remain the source the record
snapshots from. Pre-existing agents are backfilled from the live
persona once at launch, logging loudly if the linked persona is gone.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 17, 2026
The Relay E2E job (new in #939) is the first to run these previously-skipped
tests. Two asserted against `/channels/.../threads` and `/channels/.../messages`
REST routes the relay never served (permanent 404); a third raced a live
kind:44100 fan-out that a sibling subscription's drain silently discarded.

Rewrite the thread read-backs against POST /query — the depth_limit + #e
extension routes to get_thread_replies, the relay's real thread surface — and
reorder the DM test to subscribe after create_dm so the persisted membership
and discovery events are served deterministically from history.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96 wpfleger96 force-pushed the wpfleger/persona-events branch from 93a291e to 289bfc2 Compare June 17, 2026 20:03
wpfleger96 added a commit that referenced this pull request Jun 17, 2026
The Relay E2E job (new in #939) is the first to run these previously-skipped
tests. Two asserted against `/channels/.../threads` and `/channels/.../messages`
REST routes the relay never served (permanent 404); a third raced a live
kind:44100 fan-out that a sibling subscription's drain silently discarded.

Rewrite the thread read-backs against POST /query — the depth_limit + #e
extension routes to get_thread_replies, the relay's real thread surface — and
reorder the DM test to subscribe after create_dm so the persisted membership
and discovery events are served deterministically from history.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96 wpfleger96 force-pushed the wpfleger/persona-events branch from 289bfc2 to 2fb370f Compare June 17, 2026 20:28
wpfleger96 added a commit that referenced this pull request Jun 17, 2026
Persona-created agents previously re-read the live persona catalog at
every spawn, so a template edit silently changed running agents on
restart. Snapshot the persona's system_prompt/model/provider/env_vars
plus its content hash onto ManagedAgentRecord at create, and switch the
spawn and provider-deploy call sites to read that snapshot. Restart
reuses the record so it stays pinned; delete+respawn re-runs create and
rewrites the snapshot. env_vars are pinned too — without that, persona
credential edits would leak into a running agent on restart.

The shared resolve_effective_prompt_model_provider body is left intact:
its summary and ModelPicker callers intentionally read the live persona,
so only the spawn/deploy call sites move to the snapshot.

The Agents menu gets a drift indicator: build_managed_agent_summary
compares the record's pinned source_version against the persona's
current hash. A deleted persona is orphaned (never out-of-date, since
there is nothing to respawn into). The summary now displays the pinned
snapshot — what the agent actually runs — so the badge never contradicts
the fields beside it.

The mem/persona engram and fleet-update subsystem are removed: with the
record authoritative, fleet-update became a self-referential write loop
nothing read at spawn. Deletes fleet_update.rs, its launch and
persona-save triggers, the engram-at-creation write, and the
engram-only helpers in persona_events. The public persona catalog
primitives (#939) are untouched — they remain the source the record
snapshots from. Pre-existing agents are backfilled from the live
persona once at launch, logging loudly if the linked persona is gone.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 17, 2026
Persona-created agents previously re-read the live persona catalog at
every spawn, so a template edit silently changed running agents on
restart. Snapshot the persona's system_prompt/model/provider/env_vars
plus its content hash onto ManagedAgentRecord at create, and switch the
spawn and provider-deploy call sites to read that snapshot. Restart
reuses the record so it stays pinned; delete+respawn re-runs create and
rewrites the snapshot. env_vars are pinned too — without that, persona
credential edits would leak into a running agent on restart.

The shared resolve_effective_prompt_model_provider body is left intact:
its summary and ModelPicker callers intentionally read the live persona,
so only the spawn/deploy call sites move to the snapshot.

The Agents menu gets a drift indicator: build_managed_agent_summary
compares the record's pinned source_version against the persona's
current hash. A deleted persona is orphaned (never out-of-date, since
there is nothing to respawn into). The summary now displays the pinned
snapshot — what the agent actually runs — so the badge never contradicts
the fields beside it.

The mem/persona engram and fleet-update subsystem are removed: with the
record authoritative, fleet-update became a self-referential write loop
nothing read at spawn. Deletes fleet_update.rs, its launch and
persona-save triggers, the engram-at-creation write, and the
engram-only helpers in persona_events. The public persona catalog
primitives (#939) are untouched — they remain the source the record
snapshots from. Pre-existing agents are backfilled from the live
persona once at launch, logging loudly if the linked persona is gone.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 18, 2026
The Relay E2E job (new in #939) is the first to run these previously-skipped
tests. Two asserted against `/channels/.../threads` and `/channels/.../messages`
REST routes the relay never served (permanent 404); a third raced a live
kind:44100 fan-out that a sibling subscription's drain silently discarded.

Rewrite the thread read-backs against POST /query — the depth_limit + #e
extension routes to get_thread_replies, the relay's real thread surface — and
reorder the DM test to subscribe after create_dm so the persisted membership
and discovery events are served deterministically from history.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96 wpfleger96 force-pushed the wpfleger/persona-events branch from e6a44d7 to 7c98207 Compare June 18, 2026 00:56
wpfleger96 added a commit that referenced this pull request Jun 18, 2026
Persona-created agents previously re-read the live persona catalog at
every spawn, so a template edit silently changed running agents on
restart. Snapshot the persona's system_prompt/model/provider/env_vars
plus its content hash onto ManagedAgentRecord at create, and switch the
spawn and provider-deploy call sites to read that snapshot. Restart
reuses the record so it stays pinned; delete+respawn re-runs create and
rewrites the snapshot. env_vars are pinned too — without that, persona
credential edits would leak into a running agent on restart.

The shared resolve_effective_prompt_model_provider body is left intact:
its summary and ModelPicker callers intentionally read the live persona,
so only the spawn/deploy call sites move to the snapshot.

The Agents menu gets a drift indicator: build_managed_agent_summary
compares the record's pinned source_version against the persona's
current hash. A deleted persona is orphaned (never out-of-date, since
there is nothing to respawn into). The summary now displays the pinned
snapshot — what the agent actually runs — so the badge never contradicts
the fields beside it.

The mem/persona engram and fleet-update subsystem are removed: with the
record authoritative, fleet-update became a self-referential write loop
nothing read at spawn. Deletes fleet_update.rs, its launch and
persona-save triggers, the engram-at-creation write, and the
engram-only helpers in persona_events. The public persona catalog
primitives (#939) are untouched — they remain the source the record
snapshots from. Pre-existing agents are backfilled from the live
persona once at launch, logging loudly if the linked persona is gone.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 18, 2026
Persona-created agents previously re-read the live persona catalog at
every spawn, so a template edit silently changed running agents on
restart. Snapshot the persona's system_prompt/model/provider/env_vars
plus its content hash onto ManagedAgentRecord at create, and switch the
spawn and provider-deploy call sites to read that snapshot. Restart
reuses the record so it stays pinned; delete+respawn re-runs create and
rewrites the snapshot. env_vars are pinned too — without that, persona
credential edits would leak into a running agent on restart.

The shared resolve_effective_prompt_model_provider body is left intact:
its summary and ModelPicker callers intentionally read the live persona,
so only the spawn/deploy call sites move to the snapshot.

The Agents menu gets a drift indicator: build_managed_agent_summary
compares the record's pinned source_version against the persona's
current hash. A deleted persona is orphaned (never out-of-date, since
there is nothing to respawn into). The summary now displays the pinned
snapshot — what the agent actually runs — so the badge never contradicts
the fields beside it.

The mem/persona engram and fleet-update subsystem are removed: with the
record authoritative, fleet-update became a self-referential write loop
nothing read at spawn. Deletes fleet_update.rs, its launch and
persona-save triggers, the engram-at-creation write, and the
engram-only helpers in persona_events. The public persona catalog
primitives (#939) are untouched — they remain the source the record
snapshots from. Pre-existing agents are backfilled from the live
persona once at launch, logging loudly if the linked persona is gone.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96 wpfleger96 changed the title feat(desktop): add persona event kind with client publish/read/retain feat(desktop): personas-as-events foundation with config pinned at create Jun 18, 2026
wpfleger96 added a commit that referenced this pull request Jun 23, 2026
Persona create, edit, and delete now reach the relay so personas persist
across reboots and reflect in other Buzz clients. #939 built the local
retention primitives but left them inert; this wires every persona writer
to a single publish path.

pending_sync is the seam: each writer (UI create/edit, delete tombstone,
launch reconcile) retains a signed event locally with pending_sync = 1 while
holding the managed-agents store lock, and one background flush loop is the
sole relay publisher. Delete purges the persona's retention row inside the
lock-held command body (closing the same-second resurrect race) and enqueues
a NIP-09 a-tag-only kind:5 tombstone for the flush loop to publish out of
band. mark_synced is compare-and-clear on created_at+content so an edit
landing mid-flush is never falsely cleared. Ownership is intrinsic to the
signing key, so the only skip anywhere is is_builtin.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 23, 2026
Teams become authoritative on the same kind-agnostic pending_sync/flush
spine as personas (#939). A new kind:30176 NIP-33 parameterized-replaceable
event projects the public team fields (name, description, persona_ids); the
d_tag is the team id. create_team always publishes (created teams are never
built-in), update_team skips built-in teams, and delete_team enqueues an
a-tag-only kind:5 tombstone after the cascade delete — which itself rejects
built-ins, so reaching the tombstone guarantees the team was owner-published.

TeamEventContent is an explicit opt-IN projection so a future TeamRecord field
is published only when deliberately added; local install fields (source_dir,
is_symlink, is_builtin, timestamps) stay on-device.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 23, 2026
Managed agents become authoritative on the same kind-agnostic pending_sync/
flush spine as personas (#939) and teams. A new kind:30177 NIP-33
parameterized-replaceable event is owner-signed with the agent's pubkey as
d_tag, so the coordinate is 30177:<owner>:<agent_pubkey>.

ManagedAgentEventContent is an explicit opt-IN allowlist, never
record-minus-denylist: the flush loop is a blind pre-signed pipe, so this
projection is the only guard against leaking secrets to a world-readable
relay. It deliberately omits private_key_nsec, auth_tag, env_vars, the
provider backend blob, and every runtime/install field. A future record field
defaults to NOT published.

Because the retention upsert compares the serialized projection, an
operational start/stop that mutates only excluded runtime fields produces an
identical row and never re-enqueues a publish — start/stop is a no-op by
construction. Wiring follows the #939 async discipline: retain runs after
save inside the store lock with no MutexGuard held across any .await, and the
delete tombstone runs only past the deployed-remote guard so a live remote
deployment is never orphaned. Standalone agents (persona_id None) publish too
— team membership does not gate publishing.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 23, 2026
The desktop publishes team (30176) and managed-agent (30177) NIP-33
events on the #939 edit->publish spine, but the relay rejected both at
the required_scope_for_kind catch-all, so the new event kinds could
never round-trip. Register them in the same UsersWrite arm as
KIND_PERSONA (owner-authored, user-scoped state, keyed by signing key)
and mark both global-only so a stray h tag never channel-scopes them.
The persona slug envelope is not extended: team ids and 64-hex agent
pubkeys satisfy the generic parameterized-replaceable d_tag path
(D_TAG_MAX_LEN only).

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 added a commit that referenced this pull request Jun 23, 2026
Mirrors the persona e2e harness (#939) for the two new NIP-AP kinds:
publish-and-query by NIP-33 coordinate, newer-timestamp-wins
replacement, and a-tag-only tombstone-to-zero. For managed agents, one
case asserts the content round-tripped through the relay carries none
of the opt-IN-excluded fields (private_key_nsec, auth_tag, env_vars,
backend blob, runtime fields) — proving the full publish->relay->query
pipe, not just the projection function the unit tests already cover.
Team d_tags exercise the generic parameterized-replaceable d_tag path
(D_TAG_MAX_LEN only), distinct from the persona slug envelope.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wesbillman

Copy link
Copy Markdown
Collaborator

Pinky review findings

I found two CHANGE items in the consumer-side sync surface. The relay/storage/projection side looks careful overall, but another device does not reliably catch up to existing/deleted records.

CHANGE 1: Fresh consumers never backfill existing persona/team/agent events

usePersonaSync opens the sync subscription with limit: 0:

relayClient.subscribeLive(
  { kinds: PERSONA_SYNC_KINDS, authors: [pubkey], limit: 0 },
  ...
)

The relay client sends that filter as-is for the initial REQ, so a second device that starts after Device A already published records gets zero historical rows. replayLiveSubscriptions also cannot recover this first gap: lastSeenCreatedAt is undefined until a live event is seen, so reconnect replays the same limit: 0 filter. That means the PR’s stated “Device B inherits Device A’s edits” only works when Device B is already online at publish time; it does not reconstruct existing records on fresh install/start.

Suggested fix: either subscribe with a replay-capable positive limit (and page if necessary), or do an explicit one-shot history fetch for { kinds: [30175, 30176, 30177], authors: [pubkey] } before/alongside the live subscription. This path should have a regression test like observerRelay.test.mjs, which already documents that limit: 0 drops reconnect replay gaps.

Relevant code:

  • desktop/src/features/agents/lib/usePersonaSync.ts:26-31
  • desktop/src/shared/api/relayClientSession.ts:551-582
  • desktop/src/shared/api/relayReconnectReplay.ts:96-112

CHANGE 2: Deletes/tombstones are published but never consumed by desktop sync

The delete paths correctly enqueue kind:5 a-tag tombstones for personas/teams/managed agents, but the live sync subscription listens only to [30175, 30176, 30177] and reconcile_inbound_persona_event ignores every kind except those three. So if Device A deletes a persona/team/agent, the relay coordinate is soft-deleted, but Device B’s local JSON keeps the stale record forever because it never subscribes to or applies the kind:5 tombstone.

Suggested fix: include KIND_DELETION in the sync subscription and teach the reconcile command to parse a tags targeting 30175:<owner>:<d>, 30176:<owner>:<d>, and 30177:<owner>:<d>, then remove/patch the corresponding local JSON record using the same per-kind match rules. Also consider keying retained tombstones by full target coordinate (target kind + d-tag), since current tombstones are retained under (5, owner, d_tag), which can collide if a persona/team/agent share the same d-tag.

Relevant code:

  • desktop/src/features/agents/lib/usePersonaSync.ts:12,28
  • delete tombstone enqueue paths: desktop/src-tauri/src/commands/personas.rs:97-132, desktop/src-tauri/src/commands/teams.rs:88-121, desktop/src-tauri/src/commands/agents.rs:105-137
  • inbound reconcile gate/apply path: desktop/src-tauri/src/commands/personas.rs:357-423

Validation from my local review pass:

  • git diff --check origin/main...HEAD
  • bin/pnpm --dir desktop run --if-present typecheck
  • PATH="$PWD/bin:$PATH" cargo test --manifest-path desktop/src-tauri/Cargo.toml --lib retention -- --nocapture
  • PATH="$PWD/bin:$PATH" cargo test --manifest-path desktop/src-tauri/Cargo.toml --lib inbound_tests -- --nocapture
  • PATH="$PWD/bin:$PATH" cargo test --manifest-path desktop/src-tauri/Cargo.toml --lib agent_events -- --nocapture
  • PATH="$PWD/bin:$PATH" cargo test --manifest-path desktop/src-tauri/Cargo.toml --lib team_events -- --nocapture
  • PATH="$PWD/bin:$PATH" cargo test --manifest-path desktop/src-tauri/Cargo.toml --lib persona_events -- --nocapture

wpfleger96 added a commit that referenced this pull request Jun 24, 2026
The Relay E2E job (new in #939) is the first to run these previously-skipped
tests. Two asserted against `/channels/.../threads` and `/channels/.../messages`
REST routes the relay never served (permanent 404); a third raced a live
kind:44100 fan-out that a sibling subscription's drain silently discarded.

Rewrite the thread read-backs against POST /query — the depth_limit + #e
extension routes to get_thread_replies, the relay's real thread surface — and
reorder the DM test to subscribe after create_dm so the persisted membership
and discovery events are served deterministically from history.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96 wpfleger96 force-pushed the wpfleger/persona-events branch from 589435e to 633bc63 Compare June 24, 2026 20:57
wpfleger96 and others added 13 commits June 24, 2026 17:16
Squashed for rebase — original commits:
- feat(desktop): add persona event kind with client publish/read/retain
- fix(desktop): sign every migrated persona event and drop the sentinel file
- fix(relay): validate kind:30175 persona d-tag slug grammar on ingest
- fix(desktop): drop env_vars from public PersonaEventContent
- docs: add NIP-AP spec for kind:30175 persona events
- feat(ci): add relay E2E testing job and persona event tests
Missed during rebase: sprout_core→buzz_core_pkg imports in
persona_events.rs and migration.rs, sprout-desktop→buzz-desktop
in log messages, and cargo fmt on e2e_persona.rs.
cargo fmt requires buzz_core_pkg imports before nostr (alphabetical).
Relay E2E git tests need git-credential-nostr built alongside the relay.
…ay behavior

The relay now has a generic file upload path that accepts PDFs, random
bytes, and unrecognised formats as application/octet-stream downloads.
SVG with XML declaration is not detected by `infer` and also routes
through the generic path. Updated four content validation tests from
expecting rejection (400/415) to expecting acceptance (200).

Additionally, three WebSocket imeta tests hit /api/events but the
relay route is at /events (no /api prefix). Fixed the URL in all three.

Co-authored-by: Will Pfleger <wpfleger@squareup.com>
Signed-off-by: Will Pfleger <wpfleger@squareup.com>
infer detects XML-based SVG as text/xml (not image/svg+xml), which is
not in the blocked list, so the relay accepts it through the generic
file path with that MIME type.

Co-authored-by: Will Pfleger <wpfleger96@gmail.com>
Signed-off-by: Will Pfleger <wpfleger96@gmail.com>
Same class of issues as e2e_media_extended — tests assumed image-only
policy and wrong API path. The relay ignores Content-Type headers (uses
magic bytes) and the route is /events not /api/events.

Co-authored-by: Will Pfleger <wpfleger96@gmail.com>
Signed-off-by: Will Pfleger <wpfleger96@gmail.com>
The live_split_model_completes test is a manual runbook test requiring
multiple serve nodes. Replace panic!() with println+return so it skips
gracefully when CI runs --ignored tests.

Co-authored-by: Will Pfleger <wpfleger96@gmail.com>
Signed-off-by: Will Pfleger <wpfleger96@gmail.com>
The Relay E2E job (new in #939) is the first to run these previously-skipped
tests. Two asserted against `/channels/.../threads` and `/channels/.../messages`
REST routes the relay never served (permanent 404); a third raced a live
kind:44100 fan-out that a sibling subscription's drain silently discarded.

Rewrite the thread read-backs against POST /query — the depth_limit + #e
extension routes to get_thread_replies, the relay's real thread surface — and
reorder the DM test to subscribe after create_dm so the persisted membership
and discovery events are served deterministically from history.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
test_nip10_thread_reply_not_in_top_level asserted only that a reply
threads under its root — a correlate, not the relay's actual top-level
rule. get_channel_messages_top_level surfaces a depth-1 reply iff
broadcast = true, so a broadcast=1 depth-1 reply satisfied every old
assertion yet IS surfaced at top level: the test greened by data
accident. The relay exposes no top-level-queryable surface over
POST /query (feed_types routes to feed queries that never read
thread_metadata.depth/broadcast), so the rule is pinned via its two
test-observable inputs — recorded depth and the broadcast tag — in both
directions: a non-broadcast depth-1 reply is excluded, a broadcast=1
depth-1 reply is surfaced.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
cargo fmt --all --check failed Rust Lint on the assertions added in the
prior commit (lines 925, 938). Whitespace/reflow only; no assertion or
logic change.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The relay's clock-skew guard rejects events with created_at too far from
server time. Replace hardcoded Nov 2023 timestamps (1_700_000_000) with
Timestamp::now()-relative values that stay within the skew window while
preserving the older-vs-newer replacement semantics.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…2e_relay tests

Tests assumed HTTP REST endpoints (/api/events, /api/users/{pubkey}/profile,
/api/users/me/channel-add-policy) that the relay does not serve. The relay
router only exposes /events, /query, /count, and a few other paths.

Three fix patterns applied:
- POST /api/events → POST /events (the relay's actual HTTP bridge)
- GET /api/users/{pubkey}/profile → POST /query with kind:0 filter
- PUT /api/users/me/channel-add-policy → submit kind:10100 event via POST /events

Also fixed self-add test that hit nostr crate's default p-tag stripping
(EventBuilder removes p tags matching the signer unless allow_self_tagging
is called), and added a 1s sleep in kind0_nip05_sync to ensure the
replacement event gets a strictly newer created_at timestamp.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The relay-e2e job (introduced on this branch, absent on main) ran the
full `--test '*'` glob across all buzz-test-client e2e suites. That
swept in three suites targeting a REST `/api/*` surface that no longer
exists in the relay binary — the surface was migrated to the Nostr HTTP
bridge and torn out, leaving e2e_rest_api (37), e2e_tokens (19), and
e2e_workflows (7) as 63 tests that 404 against a route the relay does
not mount.

This job was added to exercise the persona/interop work, so scope it to
the two suites it covers: e2e_persona and e2e_nostr_interop. Reimplementing
the REST surface is separate, non-gating work and should not block this
merge base from going green.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 and others added 5 commits June 24, 2026 18:13
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co>
The outbound half already retains secrets-stripped persona/team/agent
projections and flushes them to the relay, but nothing read events back,
so a second device never inherited the first device's edits.

Add the inbound consumer (C1) as a sync channel that patches projected
fields INTO personas.json — retention is never an authoritative read
source, so load_personas and every agent's UUID-based resolution and
provider keys stay intact. Inbound events match the local record whose
persona_d_tag equals the event d-tag (one derivation, both directions),
so an in-app persona matches its existing UUID row instead of minting a
duplicate; the retention store resolves equal-second collisions
(preserve pending) versus strictly-newer events (inbound wins). The
frontend subscribes via the existing relayClientSession live path; a
pubkey-keyed effect re-establishes on identity change and
replayLiveSubscriptions re-fires on reconnect.

Fix two boot-order gaps (C2): run identity-independent migrations before
identity resolution via migration::run_boot_migrations, and extend the
boot content-hash reconcile to team metadata (kind 30176) alongside
personas. Delete the parked load_from_retention read primitive.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…t boot

migrate_personas_in_dir reads personas.json fresh and reconciles it into
the retention store, but it ran before sync_team_personas, which writes
team-dir edits into personas.json. So a team-dir persona edit was not
reconciled to retention — and thus did not reach the relay — until the
next launch.

Run sync before the event migrations so the writer precedes the reader
and a team-dir edit propagates same-launch. The coupling is
single-directional (sync touches only the JSON stores, migrate only
retention.db), so reordering is safe. Extracts the post-identity sync +
event-reconcile sequence into migration::run_event_sync.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
reconcile_inbound_persona_event previously no-oped on kind:30176 (team)
and kind:30177 (managed-agent), so those projections never propagated to a
second device — only 30175 personas did. Implement both inbound paths.

The managed-agent path is the secret-bearing one: it parses into the
ManagedAgentEventContent projection (10 public fields), NOT a full
ManagedAgentRecord. The projection type cannot represent private_key_nsec,
auth_tag, env_vars, backend, or agent_command/agent_command_override, so a
hostile inbound event carrying those keys has them dropped at
deserialization — the structural guard, not a post-parse filter. The apply
overwrites only the projected fields and preserves every device-local secret
and runtime field. No match is a no-op: an agent has no secret key to run
with unless it already exists locally.

The team path is symmetric on the secret discipline but inserts on no match,
since a team (like a persona) is a secretless definition another device may
legitimately learn from the relay. Local install fields (source_dir,
is_symlink, version) are preserved.

Per-kind dispatch happens before any d-tag comparison, so a d-tag shared
across kinds can never cross-link a team to a persona or agent.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Address seven review findings on the persona/team/managed-agent NIP-AP sync:

F1 — monotonic created_at. The three event builders defaulted created_at to
now; the local retention upsert replaces on >=, so a same-second second edit
was kept locally while the relay's NIP-33 lowest-id tiebreak could keep the
older event. Each retain path now reads the retained head and bumps to
max(now, head+1) per NIP-AP step 3.

F2 — consume kind:5 tombstones. Tombstones were published but never consumed,
so a delete never reached a second device. The reconcile command now handles
kind:5 (parse the a-tag coordinate, remove the local record per-kind under the
same store lock + retention spine as the apply path), and the FE subscription
covers KIND_DELETION. Retained tombstones are keyed by the full target
coordinate so cross-kind d-tag collisions occupy distinct rows.

F3 — fresh-start backfill. The live subscription used limit:0, and
reconnect-replay's since-cursor is undefined until a live event arrives, so a
device coming online after another published got zero history. The sync now
does a one-shot history fetch (including the deletion kind) before going live.

F4 — cascade persona tombstones. Deleting a directory-backed team removed its
sourced personas locally but left their kind:30175 heads live on the relay;
the cascade now returns the removed personas' d-tags and tombstones each.

F5 — d-tag normalization. Pack slugs are [a-zA-Z0-9_-]+, but the relay grammar
(matching the spec) requires ^[a-z0-9][a-z0-9_-]{0,63}$, so slugs like
CodeReviewer were rejected and stuck pending. persona_d_tag now normalizes at
the publish boundary (the spec's own write step), not by loosening the relay.

F6 — drop env_vars from the KIND_PERSONA doc-comment (spec forbids it; code was
already correct).

F7 — align PersonaEventContent field order to the NIP-AP reference vectors
(display_name, system_prompt, avatar_url, ...) so the serialized content bytes
and resulting NIP-01 event id match across implementations. Pinned by a
byte-exact vector test.

The secret-guard projection types (PersonaEventContent/TeamEventContent/
ManagedAgentEventContent) keep their exact field sets — F7 only reorders
PersonaEventContent's existing fields, no field added or removed.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96 wpfleger96 force-pushed the wpfleger/persona-events branch from 633bc63 to 71c0bcc Compare June 24, 2026 22:17
@wpfleger96 wpfleger96 enabled auto-merge (squash) June 24, 2026 22:27
@wpfleger96 wpfleger96 disabled auto-merge June 24, 2026 22:27
@wpfleger96 wpfleger96 enabled auto-merge (squash) June 24, 2026 22:27
@wpfleger96 wpfleger96 merged commit d256935 into main Jun 24, 2026
30 checks passed
@wpfleger96 wpfleger96 deleted the wpfleger/persona-events branch June 24, 2026 22:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants