Catalog My learning Blog
System Design Roadmap: 100-Day Course for Engineers · Phase 10 · Lesson 95 · 5 min read

System Design: Design a URL Shortener

On this page
  1. The problem: design a URL shortener
  2. URL shortener requirements and estimates
  3. Generating short codes, and the data model
  4. The redirect path: caching, and 301 vs 302
  5. Click analytics without slowing redirects
  6. Abuse protection and reliability
  7. What a strong URL shortener design includes
  8. Check yourself

The problem: design a URL shortener

Design a URL shortener: users submit a long URL and get a short link; visiting the link redirects. Add click analytics for link owners, custom aliases and expiry. Work through it with the habits from Days 1 to 7 before reading on. (The System Design Interview course has a shorter, interview-focused version; this one goes further into analytics and abuse.)

URL shortener requirements and estimates

Functional: create short links, redirect, custom aliases, expiry, click statistics per link. Non-functional: redirects in under 50 ms at p99 near users; very high availability for redirects; a code never maps to two URLs.

Code
New links      100 per second at peak     ≈ 3 billion per year
Redirects      100× writes                ≈ 10,000 per second, far more for viral links
Storage        3B × 500 bytes             ≈ 1.5 TB per year of link data
Clicks         10,000 events per second   ≈ 860 million click events per day

The click events, not the links, are the biggest data volume. That shapes the design. At about 200 bytes each, 860 million clicks a day is roughly 170 GB of raw events daily, over 60 TB a year, more than 40 times the link data.

Generating short codes, and the data model

  • Codes: a counter encoded in base62, handed out in blocks per server so there's no central bottleneck (Day 10); 7 characters give 3.5 trillion codes. Scramble the number before encoding if sequential codes would let people enumerate links.
  • Links table: code (primary key), long_url, owner_id, created_at, expires_at. Lookups are by key only, so a key-value store or a partitioned relational table both work; partition by code at this size (Day 26).
  • Custom aliases go into the same table with the same uniqueness constraint.

The redirect path: caching, and 301 vs 302

Redirects are read-only and links never change, so cache aggressively:

  1. CDN edge caches redirect responses for popular links (Day 35).
  2. Redis behind the service holds hot codes (Day 33); since codes are immutable, there's nothing to invalidate except expiry and takedowns.
  3. Database on a miss.

Use 302 if analytics must see every click; 301 lets browsers cache and skip you. A takedown (malicious link) purges the CDN and cache entry for that code.

Two cache details from earlier lessons apply here. Links that expire must not be cached past their expiry, so the CDN's max-age is the smaller of the usual TTL and the time left (Day 10). And requests for codes that don't exist, from typos and bots guessing codes, should get a short-lived cached 404 (Day 34), or every guess reaches the database.

Click analytics without slowing redirects

Never write to a database inside the redirect. The redirect service publishes a click event (code, time, country, referrer, device) to Kafka asynchronously and returns immediately. Stream processors aggregate counts per link per hour into an analytics store; raw events go to a data lake (Day 29). Owners see stats that are a minute or two behind, which is fine.

Filter bots before counting. Link previews in chat apps and email security scanners fetch every link they see, often before a human clicks, and would otherwise inflate every count. Tag known crawler user agents and preview fetches, and count them separately.

Redirects cached at the CDN never reach your service, so those clicks are invisible to it. If owners need exact counts, either keep CDN caching short or ingest the CDN's own request logs into the same pipeline.

URL shortener: creation through an API with block-allocated ids, redirects served from CDN and cache, and click events streamed through Kafka into analytics.
URL shortener: creation through an API with block-allocated ids, redirects served from CDN and cache, and click events streamed through Kafka into analytics.

Abuse protection and reliability

  • Rate-limit link creation per user and IP; scan new URLs against malware and phishing lists, because shorteners are a favorite tool for hiding malicious links.
  • Redirect path is stateless, active-active across zones, and leans AP: serving a cached redirect during a database problem is fine.
  • Creation path needs the strong uniqueness guarantee, and can briefly fail without affecting redirects.

What a strong URL shortener design includes

  • Numbers that reveal clicks as the dominant data volume.
  • Collision-free code generation without a single bottleneck.
  • Multi-level caching that exploits immutability.
  • Analytics moved off the redirect path through a stream, with bots filtered and CDN-served clicks accounted for.
  • Abuse prevention and a takedown path.

Check yourself

It would add latency and load to every redirect and tie redirect availability to the analytics store. Publishing an event asynchronously keeps redirects fast.

Mark the link as disabled in the database so no new lookup returns it, delete its entry from Redis, and purge that code's URL from the CDN. All three layers must be cleared, because any one of them could keep serving the redirect. Short CDN cache lifetimes for redirects limit how long a purge failure can last.

Link preview bots and security scanners. Chat apps, email filters and social networks fetch shared URLs to build previews or scan for malware, and each fetch looks like a click. Identify them by user agent, known IP ranges and behavior, and report them separately from human clicks.

A slightly stale cached redirect is harmless, but two links receiving the same code would send users to the wrong destination.