Catalog My learning Blog
System Design Roadmap: 100-Day Course for Engineers · Phase 8 · Lesson 74 · 4 min read

Retry with Exponential Backoff and Jitter

On this page
  1. How naive retries cause a retry storm
  2. Exponential backoff with jitter
  3. Which errors to retry
  4. Retry budgets and retrying at one layer
  5. Check yourself

How naive retries cause a retry storm

A downstream service has a two-second hiccup. Every caller retries immediately, then again, and again, in tight loops. The service, already struggling, now receives three or four times its normal traffic, so it can't recover, and the two-second hiccup becomes a twenty-minute outage. Retries are necessary; done carelessly, they amplify failure.

Exponential backoff with jitter

Wait longer after each failure: 0.4 s, 0.8 s, 1.6 s, 3.2 s, capped at some maximum. That's exponential backoff. It gives the struggling service time.

On its own, backoff still synchronizes callers: thousands of clients that failed at the same moment all retry at 0.4 s, then all at 0.8 s, hitting the service in waves. Add jitter, randomness in the delay, to spread them out. The "full jitter" approach picks a random delay between zero and the backoff value; AWS's architecture blog analyzed the variants and found full jitter spreads load well (Brooker, 2015).

Python
# backoff.py: retry with exponential backoff and full jitter
import random, time

RETRYABLE = {408, 429, 500, 502, 503, 504}

class HttpError(Exception):
    def __init__(self, status): self.status = status

def call_with_retries(fn, max_attempts=5, base=0.2, cap=5.0):
    for attempt in range(1, max_attempts + 1):
        try:
            return fn()
        except HttpError as e:
            if e.status not in RETRYABLE or attempt == max_attempts:
                raise                                        # 400, 401, 404...: retrying won't help
            delay = random.uniform(0, min(cap, base * 2 ** attempt))   # full jitter
            time.sleep(delay)

Against a call that failed three times with 503 and then succeeded, it waited 0.09, 0.08 and 0.63 seconds, then returned. Against a 400 it raised immediately without retrying. The ceilings for each attempt are 0.4, 0.8, 1.6, 3.2 and 5 seconds; the actual waits are random below them.

Each retry waits longer than the last, doubling the delay with random jitter, so failed callers don't retry in step.
Each retry waits longer than the last, doubling the delay with random jitter, so failed callers don't retry in step.

Which errors to retry

RetryDon't retry
Timeouts and connection errors400 Bad Request: the request is wrong
503 Service Unavailable, 502, 504401, 403: credentials won't fix themselves
429 Too Many Requests, honoring Retry-After404 Not Found
500, sometimes, for idempotent requests409 Conflict, 422 validation errors

Retry only idempotent operations, or make them idempotent first. A timed-out POST /payments may have succeeded; retrying without an idempotency key (Day 65) can charge twice.

Retry budgets and retrying at one layer

  • Cap attempts (3 to 5) and the total time, which must fit inside the caller's own deadline (Day 75).
  • Retry at one layer. If the browser retries 3 times, the gateway 3 times and the service 3 times, one failure can become 27 calls downstream.
  • Use a retry budget: allow retries only up to, say, 10 percent of normal traffic. When the budget is spent, fail fast. Combine with a circuit breaker (Day 56), which stops retries altogether once a dependency is clearly down.
  • Respect Retry-After when the server sends it.

In .NET, Polly's retry strategy supports exponential backoff with jitter out of the box; Resilience4j and most cloud SDKs offer the same. Check the defaults of whatever wraps it: .NET's AddStandardResilienceHandler() retries every HTTP method, POST included, unless you turn that off (Day 56).

Retries aren't only for HTTP. Message consumers retry too, and an immediate redelivery loop against a failing database is the same storm. Use the broker's delayed redelivery, and move a message to the dead-letter queue after a few attempts (Day 42).

Check yourself

Without jitter, every client that failed at the same moment retries at the same moments, after exactly 1, 2 and 4 seconds, creating synchronized waves of load that can knock a recovering service down again. Random delays spread the retries across each interval, so load arrives smoothly.

No. A 400 means the request itself is invalid, so sending the same request again will fail the same way and only adds load. Retry transient failures such as timeouts, 503 and 429 (respecting Retry-After), and fix the request for 4xx errors.

Jitter. Every client that failed together retries together. Randomize each delay, for example uniformly between zero and the backoff value.

Up to 3 × 3 × 3 = 27 downstream calls for one user request, which turns a small outage into a flood. Retry at only one layer, usually the outermost or the one closest to the failing dependency, cap the attempts, and use a retry budget or circuit breaker so retries stop when failures are widespread.

A limit on retries as a share of normal traffic, for example retries may add at most 10 percent extra requests. When failures spike, the budget runs out and further requests fail fast instead of multiplying load on a dependency that's already struggling.