Skip to content

fix(transport): jittered backoff for API rate limits - #4183

Open
Spazzy757 wants to merge 2 commits into
scaleway:mainfrom
Spazzy757:fix/rate-limit-jittered-backoff
Open

fix(transport): jittered backoff for API rate limits#4183
Spazzy757 wants to merge 2 commits into
scaleway:mainfrom
Spazzy757:fix/rate-limit-jittered-backoff

Conversation

@Spazzy757

Copy link
Copy Markdown

Background

We hit sustained HTTP 429 rate limiting from the Scaleway API when applying with Terraform's default concurrency.

While diagnosing, I captured a real 429 from the Scaleway API by bursting concurrent read-only requests:

HTTP/2 429
x-envoy-ratelimited: true
server: Scaleway API Gateway (fr-par-3;edge03)

Scaleway's API Gateway is Envoy-based, and its rate-limit filter returns a bare 429 with no timing guidance at all — no Retry-After, no X-RateLimit-* headers. So header-driven backoff cannot help here; client-side backoff is the only lever.

Problem

The retry transport left go-retryablehttp's DefaultBackoff in place. Its fallback exponential backoff has no jitter. Under Terraform's concurrent execution (default -parallelism=10), every throttled request backs off in lock-step (2s → 4s → 8s …) and retries at the same instant — a thundering herd that immediately re-triggers the limit.

Change

  • Replace the backoff with a jittered exponential strategy (full jitter across [min, min*2^n], capped at RetryWaitMax), wired via Client.Backoff. This spreads concurrent retries out and breaks the herd.
  • Raise RetryMax from 3 to 5 so sustained throttling gets more attempts.
  • Retry-After is still honored when present (standard, zero cost) in case an individual product API sets it — though the gateway does not.
  • Add unit tests covering Retry-After parsing, jitter bounds/variance, the x-envoy-ratelimited fallback path, the RetryWaitMax cap, and zero-min safety.

go build, go vet, gofmt, and the transport tests all pass.

🤖 Generated with Claude Code

@Spazzy757
Spazzy757 requested a review from remyleone as a code owner July 8, 2026 12:17
@Spazzy757
Spazzy757 marked this pull request as draft July 8, 2026 12:18
Scaleway's API Gateway (Envoy) returns HTTP 429 responses that carry no
timing guidance — only "x-envoy-ratelimited: true", with no Retry-After or
X-RateLimit-* headers (verified against the live API). The retry transport
left go-retryablehttp's DefaultBackoff in place, whose fallback exponential
backoff has no jitter. Under Terraform's concurrent execution, every
throttled request then backs off in lock-step and retries simultaneously,
re-triggering the limit (a thundering herd).

Replace the backoff with a jittered exponential strategy (full jitter across
[min, min*2^n], capped at RetryWaitMax) wired via Client.Backoff, and raise
RetryMax from 3 to 5. Retry-After is still honored when present in case an
individual product API sets it, though the gateway does not.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Spazzy757
Spazzy757 force-pushed the fix/rate-limit-jittered-backoff branch from 0e6a51c to 078983c Compare July 8, 2026 12:59
@Spazzy757
Spazzy757 marked this pull request as ready for review July 9, 2026 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant