🐛 fix(storage): honor reclaim-guard expiry when admitting references

[gaborbernat]

A reclaim guard carries a 300-second lease. compare_and_arm_reclaim_guards honors it, so a lapsed row is available to the next collector, but the reference-admission check in commit_driver_txn_with_journal read no clock and rejected a commit whenever a row was present. 🕒 An operator running peryx cache purge orphaned-blobs --yes arms a guard over each candidate, deletes its bytes, then disarms. Kill that loop, or let one blobs.blocking().delete return an error, and the row survives with a timestamp that lapses five minutes later. Nothing clears it. compare_and_disarm_reclaim_guard is the only removal path in the tree, and the release pass ahead of the purge disarmed an expired guard when blobs.head reported the bytes gone, which no longer holds after a failed delete or after a client re-uploads the same content.

What an operator sees depends on which path reaches the digest. A PyPI upload (POST /{index}/) and a promote (PUT .../promote) both surface MetaError through CacheError::Meta, which cache_error_status maps to 500, so the client gets metadata store error: blob <sha256> is being reclaimed; publish the reference again once its deletion finishes while no deletion is in flight; the operation ledger row stays pending, so every retry re-drives the same write and fails again. The finalize sweep logs at debug and leaves the intent pending, retrying on every tick forever. peryx import prints a rejected line per file. There is no expiry, no sweep and no admin endpoint that ends it, so the digest is unpublishable for the life of the store. One condition narrows the blast radius. The check short-circuits on an empty journal, PyPI writes a journal entry only under replication, and the purge refuses to run outside AvailabilityMode::None. Reaching the stuck state takes a purge interrupted on a node that later serves in a distributed mode.

The admission check now takes the store's clock and blocks only while !ReclaimGuard::is_expired_at(now), matching the arming side, and drops the lapsed row in the same write transaction that admits past it so an abandoned guard cannot outlive its lease. MetaStore holds the clock as an injectable peryx_core::Clock field defaulting to wall time, so a test can step the clock across a lease boundary instead of waiting one out. The release pass in purge_orphaned_blobs no longer probes the blob store. An expired lease proves its collector let go, and demanding absence was the condition that kept a failed delete's guard alive. That also retires the digest parse and the head request that existed only to answer it. Reading the guard table now checks for its existence first, because opening a table inside a redb write transaction creates it, so publishing a blob reference on a fresh store no longer materializes a distributed domain table.

Two items in the issue's required-change list are out of scope here. Distinguishing a collector guard from a claim-backed one belongs to #1383, and main has no claim-backed guard to distinguish, since compare_and_arm_reclaim_guards has one caller, the orphan purge. Reviewers of #2017 should expect a conflict in check_blob_reclaim_guard and in crates/peryx-ha-distributed/tests/unit/reclaim_guard_tests.rs; this branch sits on upstream/main and merges in either order.

Closes #1573

[codspeed-hq]

Merging this PR will not alter performance

✅ 29 untouched benchmarks
⏩ 133 skipped benchmarks¹

---

_{Comparing fix/expired-guard-blocks-references-1573 (71b9746) with main (da3120a)}

Footnotes

1. 133 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩