Skip to content

Bound the noncurrent and orphan-delete-marker listing cursors - #2696

Draft
delthas wants to merge 1 commit into
bugfix/ARSN-620/lc-phdfrom
improvement/ARSN-629/bound-lifecycle-listing-cursors
Draft

Bound the noncurrent and orphan-delete-marker listing cursors#2696
delthas wants to merge 1 commit into
bugfix/ARSN-620/lc-phdfrom
improvement/ARSN-629/bound-lifecycle-listing-cursors

Conversation

@delthas

@delthas delthas commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

DelimiterNonCurrent and DelimiterOrphanDeleteMarker stop themselves once filter() has seen maxScannedLifecycleListingEntries entries, but set no limit on the backend cursor. Everything the cursor returns past that point is read and thrown away. On a bucket whose pages return no results the budget is spent in full on every page, and behind mongos the shard cursor is opened with no batchSize and fills a 16MB batch each time: ~13,300 documents read per page against ~10,000 consumed.

Stacked on #2685 (ARSN-620), which must merge first. Based on bugfix/ARSN-620/lc-phd so the diff shows only this commit. A bound makes pages truncate on essentially every page of an orphan sweep, and truncation is only safe once the resume marker reliably advances.

The change

_cursorLimit() in DelimiterVersions, returning maxScannedLifecycleListingEntries + 1, applied in genMDParamsV0() and to both param sets in genMDParamsV1() — the two ranges are served by separate cursors and either one can supply every entry the listing consumes, so each carries the full bound rather than a share of it. Placed in the base class so both listings inherit it and the reasoning lives in one place.

No v0 doubling, and this matters. filter() counts every entry the cursor delivers, masters and versions alike, in both formats. DelimiterCurrent multiplies by 2 on v0 only because its counter lives in addContents(), which v0 version keys never reach — copying that factor here would double the bound for nothing. Its comment currently claims the counter "maps directly to cursor documents regardless of bucket format", which the ×2 block below it contradicts; corrected as a drive-by so the next person doesn't copy it.

Getting the bound wrong is not a perf regression. A cursor that ends early is indistinguishable from the end of the keyspace, and DelimiterOrphanDeleteMarker.result() flushes its held candidate whenever the scan budget was not reached — so an under-sized bound reports a delete marker as orphaned while unseen versions sit behind the cursor, and lifecycle then deletes a marker that resurrects an old version.

On the expected benefit

~25% fewer documents read per page (13,342 → 10,001 at the default budget), not the 13x ARSN-629 originally claimed. The only safe bound is the scan budget, not MaxKeys: nothing ties raw entries consumed to MaxKeys for these two listings — 10,000 single-version keys return zero results while burning the whole budget. The ticket's 13x was computed against MaxKeys + 1 = 1,001, which is not a bound these listings can honour. batchSize is the complementary lever and buys the same ~25% on this workload, since the dominant orphan-sweep pages consume the full budget; noted for a separate ticket.

Pre-existing bugs surfaced, not fixed here

Confirmed by re-running the full budget matrix with _cursorLimit() stashed — identical results, so neither is caused by this change. Both are documented in the spec where the sweep skips them:

  1. DelimiterNonCurrent cannot advance at very small budgets: resuming re-reads the marker version to recover the stale date, so the budget is consumed by skipped entries plus that re-read and NextVersionIdMarker never moves. The listing pages forever.
  2. DelimiterOrphanDeleteMarker silently loses the last orphan when the keyspace ends on the exact entry the budget ends. filter() never sees the entry that would raise FILTER_END, so IsTruncated stays false while _isMaxScannedEntriesReached() is already trueresult() then suppresses the held candidate and reports the listing complete. The orphan is dropped permanently, with no truncation to recover it. Same failure shape as the hazard this PR guards against, reached through the scan budget instead of the cursor. Worth its own ticket.

Notes for review

Issue: ARSN-629

DelimiterNonCurrent and DelimiterOrphanDeleteMarker stop themselves once
filter() has seen maxScannedLifecycleListingEntries entries, but set no
limit on the backend cursor. Everything the cursor returns past that
point is read and thrown away. On a bucket whose pages return no results
the budget is spent in full on every page, and behind mongos the shard
cursor is opened with no batchSize and fills a 16MB batch each time:
~13,300 documents read per page against ~10,000 consumed.

Bound the cursor at maxScannedLifecycleListingEntries + 1, in
DelimiterVersions so both listings inherit it. In v1 the bound goes on
BOTH param sets: the two ranges are served by separate cursors and
either one can supply every entry the listing consumes, so each carries
the full bound rather than a share of it.

The bound is +1 and carries no v0 doubling. filter() counts every entry
the cursor delivers, masters and versions alike, in both formats, and
returns FILTER_END on the entry after the budget is spent.
DelimiterCurrent multiplies by 2 on v0 only because its counter lives in
addContents(), which v0 version keys never reach; copying that factor
here would double the bound for nothing. Its comment is corrected to say
so, since it currently claims the opposite.

Getting the bound wrong is not a perf regression. A cursor that ends
early is indistinguishable from the end of the keyspace, and
DelimiterOrphanDeleteMarker.result() flushes its held candidate whenever
the scan budget was not reached -- so an under-sized bound reports a
delete marker as orphaned while unseen versions sit behind the cursor,
and lifecycle then deletes a marker that resurrects an old version.

Tests: unit coverage for the params on v0 and v1 and for the invariant
the bound rests on (the listing reaches FILTER_END within `limit`
entries), plus a functional spec paging real MongoDB listings to
exhaustion across a sweep of scan budgets. That directory had no
maxScannedLifecycleListingEntries coverage at all.

Expected effect is ~25% fewer documents read per page (13,342 -> 10,001
at the default budget), not the 13x the ticket first claimed: the only
safe bound is the scan budget, not MaxKeys.

Issue: ARSN-629
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.76%. Comparing base (be3afa4) to head (c0cd144).

Additional details and impacted files
@@                   Coverage Diff                   @@
##           bugfix/ARSN-620/lc-phd    #2696   +/-   ##
=======================================================
  Coverage                   74.75%   74.76%           
=======================================================
  Files                         229      229           
  Lines                       18715    18726   +11     
  Branches                     3898     3871   -27     
=======================================================
+ Hits                        13991    14001   +10     
- Misses                       4719     4720    +1     
  Partials                        5        5           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant