Skip to content

card-action dispatch: token-guard the RouteMissingSeenAt marker write (symmetric to ClearRouteMissing)Β #680

Description

@an9xyz

Follow-up from the PR #662 review (yujiawei, P2; Jerry-Xin concurred). Non-blocking defense-in-depth.

Problem

In internal/cardactiondispatch/queue.go, the route-missing first-seen marker (route_missing_since) is now cleared under a token guard (clearRouteMissingScript: HGET tokens[id] == token before HDEL), but it is written without one β€” routeMissingSinceScript (RouteMissingSeenAt) does HSETNX keyed only by event_id, no ownership check.

This leaves one asymmetric-refund corner open. Trace:

  1. Worker A (token T-A) claims a route-missing event and issues RouteMissingSeenAt, but the Redis call stalls in flight (GC/scheduler).
  2. A's lease expires β†’ ReclaimExpired requeues it β†’ Worker B (token T-B) claims it. B sees the route has recovered, calls ClearRouteMissing (valid T-B) β†’ marker gone β†’ B proceeds to Deliver.
  3. A's stalled RouteMissingSeenAt finally lands and re-plants the marker (HSETNX on the now-absent field).
  4. B hard-crashes mid-delivery (no Ack/Nack) β†’ reclaimScript sees the re-planted marker β†’ refunds B's genuine delivery attempt β€” the exact MaxAttempts-bound bypass PR fix(cardaction): harden route-missing defer path β€” reclaim attempt-leak + LiveTTL floor (#623, #624)Β #662 closed for the common path.

Severity: astronomically narrow / non-blocking

Requires an in-flight marker write reordered past a lease expiry and a reclaim + re-claim and a route recovery and a delivery crash, all in the same episode. No data loss (idempotent consumer); worst case is a delayed dead-letter. Reviewers explicitly called this a follow-up, not a merge blocker.

Possible fix

Token-guard the marker write symmetric to the clear: give routeMissingSinceScript the tokens key + token arg and only HSETNX when HGET tokens[id] == token. This is a signature change β€” RouteMissingSeenAt gains a token param β€” touching the dispatchQueue interface and the two test fakes; the dispatcher already holds lease.Token at the call site.

Ref: PR #662, yujiawei review at head fe386b40; tracked in .octospec/tasks/card-action-dispatch-defer-hardening/context.yaml as item (c).

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions