fix(transport): jittered backoff for API rate limits - #4183
Open
Spazzy757 wants to merge 2 commits into
Open
Conversation
Scaleway's API Gateway (Envoy) returns HTTP 429 responses that carry no timing guidance — only "x-envoy-ratelimited: true", with no Retry-After or X-RateLimit-* headers (verified against the live API). The retry transport left go-retryablehttp's DefaultBackoff in place, whose fallback exponential backoff has no jitter. Under Terraform's concurrent execution, every throttled request then backs off in lock-step and retries simultaneously, re-triggering the limit (a thundering herd). Replace the backoff with a jittered exponential strategy (full jitter across [min, min*2^n], capped at RetryWaitMax) wired via Client.Backoff, and raise RetryMax from 3 to 5. Retry-After is still honored when present in case an individual product API sets it, though the gateway does not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Spazzy757
force-pushed
the
fix/rate-limit-jittered-backoff
branch
from
July 8, 2026 12:59
0e6a51c to
078983c
Compare
Spazzy757
marked this pull request as ready for review
July 9, 2026 13:23
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
We hit sustained HTTP 429 rate limiting from the Scaleway API when applying with Terraform's default concurrency.
While diagnosing, I captured a real 429 from the Scaleway API by bursting concurrent read-only requests:
Scaleway's API Gateway is Envoy-based, and its rate-limit filter returns a bare 429 with no timing guidance at all — no
Retry-After, noX-RateLimit-*headers. So header-driven backoff cannot help here; client-side backoff is the only lever.Problem
The retry transport left
go-retryablehttp'sDefaultBackoffin place. Its fallback exponential backoff has no jitter. Under Terraform's concurrent execution (default-parallelism=10), every throttled request backs off in lock-step (2s → 4s → 8s …) and retries at the same instant — a thundering herd that immediately re-triggers the limit.Change
[min, min*2^n], capped atRetryWaitMax), wired viaClient.Backoff. This spreads concurrent retries out and breaks the herd.RetryMaxfrom 3 to 5 so sustained throttling gets more attempts.Retry-Afteris still honored when present (standard, zero cost) in case an individual product API sets it — though the gateway does not.Retry-Afterparsing, jitter bounds/variance, thex-envoy-ratelimitedfallback path, theRetryWaitMaxcap, and zero-min safety.go build,go vet,gofmt, and the transport tests all pass.🤖 Generated with Claude Code