[GG] fix(kv-offload): defer stores until metadata is ready

[voipmonitor]

Summary

• bound each KV group's store cursor by the common ready prefix of token eligibility, hash keys, and complete tracked GPU-block chunks
• retry a lagging chunk on a later scheduler step instead of asserting or advancing past it
• retain the key/block length assertion after the readiness bound as an internal consistency guard

Root cause

With async scheduling, token-derived store progress can become visible before either offload_keys or block_ids reaches the same chunk boundary. The old code sliced both collections at the token-derived boundary and asserted equal lengths, which could terminate EngineCore during queue abort, eviction, or preemption.

Removing the assertion or relying on zip() is not sufficient: advancing next_stored_chunk_idx past the missing metadata would silently skip that chunk forever. storable_chunks() now owns the complete invariant and returns only the common ready prefix.

Relation to earlier work

This replaces the closed #133, which targeted a transient CUTLASS integration branch instead of dev/gilded-gnosis.

The implementation was re-audited against upstream #49146:

• it follows the upstream-reviewed design by keeping readiness inside storable_chunks()
• it includes the upstream queued-abort behavior
• it additionally gates on available hash keys, since keys and block IDs can lag independently
• it covers multi-block chunks and verifies that deferred chunks are retried

Validation

Run in the GG release CUDA image:

• focused metadata-lag and queued-abort cases: 6 passed
• complete test_scheduler.py: 97 passed
• complete tests/v1/kv_connector/unit/offloading_connector: 142 passed
• Ruff lint and format checks passed
• git diff --check passed

The branch is based directly on dev/gilded-gnosis at 73e4a8cde8; there is no stacked base or Docker-only overlay.

[coderabbitai]

Warning

Review limit reached

@voipmonitor, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 15 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 86c84a8c-865f-4c71-ab36-a045837009bf

📥 Commits

Reviewing files that changed from the base of the PR and between 73e4a8c and 282dde3.

📒 Files selected for processing (2)

• tests/v1/kv_connector/unit/offloading_connector/test_scheduler.py
• vllm/distributed/kv_transfer/kv_connector/v1/offloading/scheduler.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/gg-kv-offload-metadata-ready-20260721

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}