Skip to content

🐛 fix(availability): prove reclaim guards by reference revision - #2017

Merged
gaborbernat merged 1 commit into
mainfrom
fix/guard-references-through-deletion-1383
Aug 31, 2026
Merged

🐛 fix(availability): prove reclaim guards by reference revision#2017
gaborbernat merged 1 commit into
mainfrom
fix/guard-references-through-deletion-1383

Conversation

@gaborbernat

Copy link
Copy Markdown
Member

The collector that purges orphaned blobs arms a per-digest reference guard around its delete, and both halves of that guard proved themselves against the replication journal. Neither half sees the commit it exists to catch.

ReclaimGuardStore::reclaim_guard_serial returned MetaStore::current_serial, and compare_and_arm_reclaim_guards compared that journal serial inside its arming transaction. commit_journal advances the serial only for a non-empty journal list, and a driver transaction can add a reference without appending an entry, which #2012 established for the readiness verdict. journal_entries returns nothing whenever the outbox is off, so on a deployment without replication a file published between the collector's reference scan and its arm left the serial where it was and the arm succeeded over a digest that had just gained a reference. check_blob_reclaim_guard read the same signal from the other side and returned early on an empty journal, so on that same deployment an armed guard rejected nothing and a publish landing between the arm and backend.delete committed through it.

Arming now carries the reference revision every driver-row write advances, and the store re-reads that revision inside the arming transaction. ReclaimGuardArm::ReferencesMoved retires a scan the way TombstoneWrite::ReferencesMoved retires a verdict, and the collector re-scans rather than guarding a digest that is no longer orphaned. The admission check reads the transaction's declared blob set instead of the last journal entry's, so it rejects a reference commit whether or not that commit replicates. A replica applying the primary's journal still passes, because it carries an expected serial and the primary fenced that write against its own guards before journaling it. Opening a table in a write transaction creates it, so the check asks whether the guard table exists first and a store that never armed a guard still grows no table.

flowchart LR
    R0[Read reference revision] --> Scan[Scan owner references]
    Scan --> Arm{Revision still held?}
    Arm -- no --> R0
    Arm -- yes --> Guard[Arm guard, reject new references]
    Guard --> Delete[Delete bytes]
    Delete --> Disarm[Disarm guard]
    classDef accent fill:#cfe4ff,stroke:#1f6feb,color:#0b1f3a;
    classDef warn fill:#ffe3a3,stroke:#d29200,color:#3a2c00;
    class R0,Scan,Guard,Delete,Disarm accent;
    class Arm warn;
Loading

One point of the issue stayed out. The Ready-tombstone executor it also asks for does not exist to guard, since reclaim_pass marks tombstones Ready and stops. The reclamation page now says bytes stay until a collector claims the digest rather than promising an executor that removes them, which is the alternative the third acceptance criterion allows. Carrying a delete-claim generation into a conditional-delete backend needs that executor first.

Closes #1383

@gaborbernat gaborbernat added the bug Something isn't working label Aug 31, 2026
@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 peryx | 🛠️ Build #34319565 | 📁 Comparing 3848d54 against latest (da3120a)

  🔍 Preview build  

1 file changed
± core/availability/blob-reclamation/index.html

@codspeed-hq

codspeed-hq Bot commented Aug 31, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 29 untouched benchmarks
⏩ 133 skipped benchmarks1


Comparing fix/guard-references-through-deletion-1383 (025f54a) with main (68a4b98)

Open in CodSpeed

Footnotes

  1. 133 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@gaborbernat
gaborbernat force-pushed the fix/guard-references-through-deletion-1383 branch from 3848d54 to 7224856 Compare August 31, 2026 22:45
The collector that purges orphaned blobs arms a per-digest reference guard
around its delete, and both halves of that guard proved themselves against
the replication journal. A driver transaction can add a reference without
appending a journal entry, which is every publish on a deployment without
replication, so the arm compared a serial that had not moved and the
admission check returned early on an empty journal. A file published across
the destructive interval lost its bytes.

Arming now carries the reference revision every driver-row write advances,
and the store re-reads it inside the arming transaction. The admission check
reads the transaction's declared blob set instead of the last journal
entry's, so it rejects a reference commit whether or not that commit
replicates. Opening a table in a write transaction creates it, so the check
asks whether the guard table exists first and a store that never armed a
guard still grows no table.

Closes #1383
@gaborbernat
gaborbernat force-pushed the fix/guard-references-through-deletion-1383 branch from 7224856 to 025f54a Compare August 31, 2026 22:45
@gaborbernat
gaborbernat merged commit 80f8eb7 into main Aug 31, 2026
27 checks passed
@gaborbernat
gaborbernat deleted the fix/guard-references-through-deletion-1383 branch August 31, 2026 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Guard references and fences through blob deletion

1 participant