Catalog My learning Blog
Backend for Frontend (BFF) Course with Node and Fastify · Phase 3 · Lesson 10 · 6 min read

Retries, Circuit Breakers and Caching in a BFF

On this page
  1. Retry only what's safe to repeat
  2. Stop calling a service that's down
  3. Cache what's safe to cache
  4. Check yourself

Retry only what's safe to repeat

Many failures are brief: a pod restarting, a dropped connection, a 503 during a deploy. One quick retry often succeeds. But retries are dangerous when used carelessly:

  • Retrying a non-idempotent call, like POST /payments, can do the work twice if the first attempt actually succeeded and only the response was lost.
  • Retrying non-transient errors, like 400 or 403, never helps.
  • Retrying immediately and in lockstep turns a struggling service into a dead one, as every client hammers it at the same moment.

A small helper encodes those rules:

TypeScript
async function withRetry<T>(fn: () => Promise<T>, { attempts = 3, baseMs = 100 } = {}): Promise<T> {
  for (let i = 1; ; i++) {
    try {
      return await fn();
    } catch (err: any) {
      const retryable = err.name === 'TimeoutError' || [502, 503, 504].includes(err.status);
      if (!retryable || i >= attempts) throw err;
      const delay = baseMs * 2 ** (i - 1) * (0.5 + Math.random());   // backoff with jitter
      await new Promise((r) => setTimeout(r, delay));
    }
  }
}

const recs = await withRetry(() => getJson(`${RECS_API}/recs`, { timeoutMs: 300 }));

Use it for GET requests and for writes that the service makes idempotent, typically with an Idempotency-Key header. Against a service that failed twice and then recovered, the lab got the data on the third call. The exponential backoff (100, 200, 400 ms) gives the service room, and the jitter spreads clients out so they don't all retry together.

Mind the total time. Three attempts with a 300 ms timeout plus backoff can take over a second, which may blow the endpoint's own deadline from Day 9. Retries must fit inside it.

Stop calling a service that's down

When a dependency is truly down, retries make things worse. Every request waits for timeouts, ties up the BFF and keeps hitting a service that's trying to recover. A circuit breaker watches the failure rate and, past a threshold, stops calling the service for a while:

  • Closed: calls go through normally, and failures are counted.
  • Open: calls fail immediately, without touching the service, and a fallback answers instead.
  • Half-open: after a pause, one trial call is allowed. If it succeeds, the breaker closes. If not, it opens again.
A circuit breaker moves between closed, open and half-open states.
A circuit breaker moves between closed, open and half-open states.

opossum is a widely used circuit breaker for Node:

JavaScript
import CircuitBreaker from 'opossum';

const recsBreaker = new CircuitBreaker(() => getJson(`${RECS_API}/recs`, { timeoutMs: 300 }), {
  timeout: false,                  // getJson already has its own timeout
  errorThresholdPercentage: 50,    // open when half of recent calls fail...
  volumeThreshold: 4,              // ...once at least 4 calls have been seen
  resetTimeout: 10_000,            // after 10 s, allow one trial call
});
recsBreaker.fallback(() => ({ items: [], fallback: true }));
recsBreaker.on('open', () => log.warn('recommendations circuit opened'));
recsBreaker.on('close', () => log.info('recommendations circuit closed'));

const recs = await recsBreaker.fire();

The lab ran eight calls against a service returning 503, with a 1-second reset for the test:

Code
breaker while down:  fallback fallback fallback fallback(open) fallback(open) fallback(open) fallback(open) fallback(open)
downstream calls:    4
after reset timeout: ok    transitions: open -> halfOpen -> close

The first four calls reached the service and failed. The breaker opened, and the next four were answered by the fallback without any network call. Once the service recovered, the trial call succeeded and the breaker closed.

Keep one breaker per dependency per BFF instance, shared by all requests, not one per request. Combined with Day 9's degraded list, the dashboard stays fast while recommendations are down, and the recommendations team gets a break to recover.

Cache what's safe to cache

Some downstream data changes slowly: feature flags, a product catalog, a user's profile. Caching it in the BFF cuts latency and load. The risk in a BFF is serving one user's data to another, so put the user in the cache key for anything personal:

TypeScript
const cache = new Map<string, { value: unknown; until: number }>();

async function cached<T>(key: string, ttlMs: number, load: () => Promise<T>): Promise<T> {
  const hit = cache.get(key);
  if (hit && hit.until > Date.now()) return hit.value as T;
  const value = await load();
  cache.set(key, { value, until: Date.now() + ttlMs });
  return value;
}

const recs = await cached(`recs:${session.data.user.sub}`, 60_000, () => recsBreaker.fire());

In the lab, three reads (two for Ada, one for Bob) made two downstream calls: Ada's second read came from the cache, and Bob got his own entry.

For production, bound the cache's size (an LRU cache such as lru-cache) or move it to Redis so instances share it. Keep TTLs short for anything a user can change, and delete the entry when the BFF itself makes a change, like after creating a task.

Be careful with HTTP caching headers on BFF responses. A personal response must carry Cache-Control: private, no-store (or at least private), so a CDN or shared proxy never stores it and hands it to the next user.

TipEvery resilience feature should be visible. Count retries, breaker state changes, fallbacks and cache hits in your metrics. A breaker that opens silently every afternoon is a problem you'd want to know about.

Check yourself

The first attempt may have succeeded on the server even though the response was lost, so a retry could charge twice. Retry only idempotent calls, or writes made idempotent with a key.

They fail fast without calling the service, and a fallback answers. After the reset timeout, one trial call decides whether the breaker closes again.

Recommendations are personal. If the key were just recs, the first user's results would be cached and served to everyone after them. Put the user id in the key, keep the time to live short, and mark personal responses Cache-Control: private so no CDN or shared proxy stores them.

Every client retries immediately and at the same moment, multiplying the load on a service that's already struggling, which can keep it from recovering. Exponential backoff spaces attempts out, and jitter spreads clients apart so they don't retry in lockstep.

An open breaker doesn't call the service at all until its reset timeout passes. Then it lets one trial call through in the half-open state, and only closes again if that call succeeds. That pause is deliberate: it gives a recovering service room instead of hitting it with full traffic immediately.