Catalog My learning Blog
System Design Roadmap: 100-Day Course for Engineers · Phase 4 · Lesson 37 · 4 min read

Session Management at Scale

On this page
  1. Symptom: random logouts after adding servers
  2. Option 1: sticky sessions
  3. Option 2: a shared session store in Redis
  4. Option 3: stateless JWT tokens
  5. The common hybrid: access and refresh tokens
  6. Check yourself

Symptom: random logouts after adding servers

An ASP.NET Core app ran on one server using the default in-memory session. Traffic grew, two more servers went in behind a load balancer, and users started reporting random logouts and empty carts. Each server only knew the sessions it had created. Day 2 named the problem; this lesson covers the options for fixing it.

Option 1: sticky sessions

The load balancer remembers which server a user hit first, usually with a cookie, and keeps sending them there. No code changes, which is why teams reach for it first.

It doesn't hold up:

  • When that server dies or is restarted during a deploy, its users lose their sessions.
  • Load gets uneven: a few heavy users can pin one server.
  • Autoscaling down kills sessions on removed instances.

Fine as a stopgap; not a design.

Option 2: a shared session store in Redis

Every server reads and writes sessions in one shared store, usually Redis, sometimes the database. Any server can handle any request. In ASP.NET Core it's a configuration change: register a Redis-backed distributed cache, and the session middleware uses it. Most frameworks have the equivalent.

Things to get right:

  • Expiry: a TTL per session, with sliding expiration (each request extends it) and an absolute maximum.
  • Size: keep sessions small (ids and a few flags), not whole object graphs. Every request reads them.
  • Availability: the session store is now on every request's path. Run it with replication and failover (Day 25), or a Redis outage logs everyone out.
  • Revocation: logging out or "sign out everywhere" means deleting the session, which takes effect immediately.

Option 3: stateless JWT tokens

Put the user's identity and claims in a signed token, typically a JWT, that the client sends on every request. Any server verifies the signature and reads the claims without looking anything up. No session store at all.

The catch is revocation. A token is valid until it expires, even after the user logs out or is disabled, because no server-side record exists to delete. Large tokens also travel with every request: browsers cap a cookie at about 4 KB, which is why ASP.NET Core splits big auth cookies into several chunks.

Where the browser keeps the token matters as much as what's in it. A token in localStorage can be read by any script on the page, so one cross-site scripting bug leaks every user's token. An HttpOnly, Secure, SameSite cookie can't be read by scripts, but needs CSRF protection. For single-page apps, the safest common pattern keeps tokens out of the browser entirely: a backend-for-frontend (Day 18) holds them server-side and the browser gets only a session cookie.

Sticky sessions pin a user to one server, a shared Redis store lets any server serve any user, and JWTs remove server-side session storage.
Sticky sessions pin a user to one server, a shared Redis store lets any server serve any user, and JWTs remove server-side session storage.

The common hybrid: access and refresh tokens

Most systems combine the last two:

  • A short-lived access token (5 to 15 minutes) for fast, stateless checks on every API call.
  • A refresh token stored server-side, used to get new access tokens. Revoking it ends the session within one access token lifetime.
  • Server-side state, such as a cart, kept in Redis or the database, keyed by user id rather than squeezed into a token.
StickyShared storeStateless JWT
Survives a server failureNoYesYes
Lookup per requestNoYesNo
Immediate revocationYesYesNo (wait for expiry)
Extra infrastructureNoSession storeKey management

One more thing on ASP.NET Core: even with sessions in Redis, auth cookies are encrypted with Data Protection keys, which must also be shared across servers (Day 2). Both have to move, or users still get logged out.

Check yourself

Any injected script can read and steal it. Keep tokens server-side in a BFF and give the browser an HttpOnly, SameSite session cookie.

JWTs are valid until they expire, since there's no server-side session to delete. Short access-token lifetimes plus revocable refresh tokens limit the window.

Replication with automatic failover, such as Sentinel or a managed service's high availability, plus monitoring and alerting, sensible client timeouts and retries, and capacity headroom. Every authenticated request now depends on the session store, so its outage is a site-wide outage.

As a short-term measure for a legacy app that keeps state in memory. They break when a server dies or is redeployed, since its users lose their sessions, and they make load uneven. Moving state to a shared store is the long-term fix.

A short access token, such as 10 minutes, limits the damage if it leaks, while the refresh token lets the client get new ones without a new login. The server can revoke refresh tokens, giving you a way to end sessions that a stateless JWT alone lacks.