🐛 fix(availability): prove reclaim guards by reference revision - #2017
Merged
Conversation
Merging this PR will not alter performance
Comparing Footnotes
|
gaborbernat
force-pushed
the
fix/guard-references-through-deletion-1383
branch
from
August 31, 2026 22:45
3848d54 to
7224856
Compare
The collector that purges orphaned blobs arms a per-digest reference guard around its delete, and both halves of that guard proved themselves against the replication journal. A driver transaction can add a reference without appending a journal entry, which is every publish on a deployment without replication, so the arm compared a serial that had not moved and the admission check returned early on an empty journal. A file published across the destructive interval lost its bytes. Arming now carries the reference revision every driver-row write advances, and the store re-reads it inside the arming transaction. The admission check reads the transaction's declared blob set instead of the last journal entry's, so it rejects a reference commit whether or not that commit replicates. Opening a table in a write transaction creates it, so the check asks whether the guard table exists first and a store that never armed a guard still grows no table. Closes #1383
gaborbernat
force-pushed
the
fix/guard-references-through-deletion-1383
branch
from
August 31, 2026 22:45
7224856 to
025f54a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The collector that purges orphaned blobs arms a per-digest reference guard around its delete, and both halves of that guard proved themselves against the replication journal. Neither half sees the commit it exists to catch.
ReclaimGuardStore::reclaim_guard_serialreturnedMetaStore::current_serial, andcompare_and_arm_reclaim_guardscompared that journal serial inside its arming transaction.commit_journaladvances the serial only for a non-empty journal list, and a driver transaction can add a reference without appending an entry, which #2012 established for the readiness verdict.journal_entriesreturns nothing whenever the outbox is off, so on a deployment without replication a file published between the collector's reference scan and its arm left the serial where it was and the arm succeeded over a digest that had just gained a reference.check_blob_reclaim_guardread the same signal from the other side and returned early on an empty journal, so on that same deployment an armed guard rejected nothing and a publish landing between the arm andbackend.deletecommitted through it.Arming now carries the reference revision every driver-row write advances, and the store re-reads that revision inside the arming transaction.
ReclaimGuardArm::ReferencesMovedretires a scan the wayTombstoneWrite::ReferencesMovedretires a verdict, and the collector re-scans rather than guarding a digest that is no longer orphaned. The admission check reads the transaction's declared blob set instead of the last journal entry's, so it rejects a reference commit whether or not that commit replicates. A replica applying the primary's journal still passes, because it carries an expected serial and the primary fenced that write against its own guards before journaling it. Opening a table in a write transaction creates it, so the check asks whether the guard table exists first and a store that never armed a guard still grows no table.flowchart LR R0[Read reference revision] --> Scan[Scan owner references] Scan --> Arm{Revision still held?} Arm -- no --> R0 Arm -- yes --> Guard[Arm guard, reject new references] Guard --> Delete[Delete bytes] Delete --> Disarm[Disarm guard] classDef accent fill:#cfe4ff,stroke:#1f6feb,color:#0b1f3a; classDef warn fill:#ffe3a3,stroke:#d29200,color:#3a2c00; class R0,Scan,Guard,Delete,Disarm accent; class Arm warn;One point of the issue stayed out. The
Ready-tombstone executor it also asks for does not exist to guard, sincereclaim_passmarks tombstonesReadyand stops. The reclamation page now says bytes stay until a collector claims the digest rather than promising an executor that removes them, which is the alternative the third acceptance criterion allows. Carrying a delete-claim generation into a conditional-delete backend needs that executor first.Closes #1383