Retry storms: why exponential backoff isn’t enough without jitter

A service has a brief outage: a burst of requests fails. Well-written clients all retry, and that very behaviour can turn a short outage into a long one. When many clients retry at the same moment, the freshly recovered server is flooded again and falls over, a phenomenon called a retry storm or thundering herd. Exponential backoff helps, but it isn't enough: you also need jitter.

Why exponential backoff isn't enough

Exponential backoff increases the wait between attempts (e.g. 0.8s, 1.6s, 3.2s…). But if 1,000 clients fail at the same moment, they also all wait 0.8s and retry together: their clocks stay synchronised, creating repeated "waves" of load.

The scenario in the video

Same number of retries; only the timing of the retries differs.

Sample code

function delay(attempt, base = 200, cap = 10000) {
  const max = Math.min(cap, base * 2 ** attempt);
  return Math.random() * max;           // full jitter
}
async function withRetry(fn, retries = 5) {
  for (let i = 0; ; i++) {
    try { return await fn(); }
    catch (e) {
      if (i >= retries || !isTransient(e)) throw e;
      await new Promise(r => setTimeout(r, delay(i)));
    }
  }
}

Jitter variants

• Full jitter: wait a random time within [0, backoff]; spreads load most evenly.
• Equal jitter: half fixed plus half random; guarantees a minimum wait.
• Decorrelated jitter: the wait depends on the previous wait, avoiding long-term synchronisation.

Doing it right

• Always combine: exponential backoff + jitter + a retry limit + an overall timeout.
• Only retry transient errors (503, timeouts, connection loss); don't retry 4xx errors caused by bad requests.
• Ensure idempotency for retried operations (use idempotency keys for payment APIs, order creation…).
• Combine with a circuit breaker to stop retrying altogether when the target service is down.
• Honour Retry-After if the server sends it.

Check your code

1. Find every hand-written retry loop.
2. Check for jitter, limits and a distinction between transient and permanent errors.
3. Simulate a brief outage with a load test and watch the shape of the load.