Design URL Shortener

Problem Statement

Design URL Shortener as a production service where growth exposes the first weak assumption in the original design. For Design URL Shortener, name the critical user action, the data that must remain correct, and the first bottleneck that appears when traffic grows.

Deep-Dive: Clarifying Questions for URL Shortener

Before designing, a senior candidate asks these SPECIFIC questions:

1. What is the read-to-write ratio? URL shorteners are extremely read-heavy — typically 100:1 (for every URL created, it is redirected 100 times). This fundamentally shapes our architecture.
2. How long should short URLs live? Permanent (like bit.ly links) or with a TTL (like temporary marketing campaign links)? Permanent means storage grows forever and we need to plan for that.
3. Do we need custom aliases? Can users choose "mysite.co/summer-sale" or only get random short codes? Custom aliases require uniqueness checking against the entire key space.
4. Do we need analytics? Click counts, geographic distribution, referrer tracking, time-series click data? Analytics significantly increases storage and compute requirements.
5. What is the maximum URL length we accept? Most browsers support URLs up to 2,048 characters, but some systems allow longer. This affects our storage sizing.
6. Should URLs be guessable? Sequential IDs (abc123, abc124) let attackers enumerate all URLs. Random 7-character base62 codes make enumeration impractical.

Specific Functional Requirements

1. URL Shortening: Given a long URL, generate a unique short URL (e.g., short.ly/aB3xK9) that redirects to the original
2. URL Redirection: When a user visits the short URL, redirect them to the original URL with HTTP 301 (permanent) or 302 (temporary) redirect
3. Custom Aliases: Optionally allow users to specify a custom short code (e.g., short.ly/my-brand) with uniqueness validation
4. Link Expiration: Support optional TTL — links can expire after a specified date
5. Analytics Dashboard: Track click count, unique visitors, geographic distribution, and referrer for each short URL
6. API Access: Provide a REST API for programmatic URL creation and analytics retrieval

Specific API Endpoints

POST /api/v1/urls
  Body: { "long_url": "https://example.com/very/long/path", "custom_alias": "my-link", "expires_at": "2025-12-31" }
  Response: { "short_url": "https://short.ly/aB3xK9", "long_url": "...", "created_at": "...", "expires_at": "..." }

GET /:short_code
  Response: HTTP 301 Redirect to long_url
  Headers: Location: https://example.com/very/long/path

GET /api/v1/urls/:short_code/stats
  Response: { "total_clicks": 15234, "unique_visitors": 8921, "clicks_by_country": {...}, "clicks_by_day": [...] }

DELETE /api/v1/urls/:short_code
  Response: HTTP 204 No Content

Specific Data Model

URLs Table (Primary Storage)

Click Events Table (Analytics)

Use a NoSQL store (Cassandra or DynamoDB) for click events due to the write-heavy, append-only nature. Use a relational database (PostgreSQL) for the URL mapping for strong consistency on reads.

Specific Back-of-the-Envelope Numbers

Traffic Estimates:

• 100M new URLs created per month (similar to bit.ly scale)
• Read:Write ratio of 100:1 means 10B redirections per month
• Writes: 100M / (30 days * 24 hours * 3600 seconds) = ~40 writes/second
• Reads: 40 * 100 = ~4,000 reads/second (average), peak at 3-5x = ~15,000 reads/second

Storage Estimates:

• Short code: 7 characters (base62 = 62^7 = 3.5 trillion possible URLs — enough for decades)
• Average URL mapping: 7 bytes (code) + 200 bytes (avg long URL) + 50 bytes (metadata) = ~257 bytes
• 100M new URLs/month * 12 months * 5 years = 6B URLs = 6B * 257 bytes = ~1.5 TB for URL mappings
• Click events: much larger, partitioned by time, older data moved to cold storage

Bandwidth:

• Each redirect: ~500 bytes (HTTP headers) = 4,000 * 500 = 2 MB/second (negligible)

Caching:

• 80/20 rule: 20% of URLs generate 80% of traffic
• Cache the top 20% of URLs: 6B * 0.2 * 257 bytes = ~300 GB (fits in a Redis cluster)

Sources

• Design URL Shortener — Reference
• Source: System-Design-Overview

URL Shortener Interview Deep Dive

Interviewers use the URL shortener problem to test fundamentals, but the questions they ask go far deeper than most candidates expect. Here are the exact questions and the answers that demonstrate senior-level thinking.

"How do you generate unique short codes?" There are three main approaches, each with different tradeoffs. (1) Base62 encoding of an auto-incrementing counter: a centralized counter (or database sequence) generates sequential IDs, which you encode to base62 (a-z, A-Z, 0-9). Short code "aB3xK9" maps to a specific integer. Pros: guaranteed unique, compact, predictable length. Cons: the counter is a single point of failure and a bottleneck. (2) MD5/SHA256 hash of the URL, then take the first 7 characters: distributed and stateless, but collisions are possible and must be handled. Bitly uses a variant of this approach. (3) Pre-generated key service: a background process pre-generates millions of unique short codes and stores them in a key pool. When a new URL is shortened, the service grabs the next available key. This eliminates collision checking at write time and is the most scalable approach for high write volumes.

"How do you handle collisions?" For hash-based generation, collisions are inevitable. When a collision is detected (the generated short code already exists in the database), append a retry counter or salt to the input and rehash. Use an exponential backoff or a maximum retry limit. Alternatively, switch to the pre-generated key approach where collisions are impossible by design because each key is only used once. For the counter-based approach, collisions do not occur because each counter value is unique — but you must handle the case where two instances try to use the same counter value concurrently, typically solved with a distributed lock or partitioned counter ranges (each server owns a range of counter values).

"How would you scale to 1 billion URLs?" 1 billion URLs at roughly 300 bytes each is about 300 GB — this fits on a single database server with a good SSD. The challenge is not storage but read throughput. With a 100:1 read-to-write ratio, 1 billion URLs could mean 100 billion redirects, or roughly 3,000 reads per second on average. This is manageable with a single PostgreSQL instance and a Redis cache. But at peak, you might see 30,000+ reads per second. The scaling strategy: (1) Put a Redis cluster in front of the database caching the most popular URLs — the top 20% of URLs serve 80% of traffic. (2) Use database read replicas for cache misses. (3) Shard the database by short code using consistent hashing if a single instance cannot handle the write load. (4) Use a CDN to cache redirects for extremely popular URLs at the edge. Companies like Bitly handle 600 million clicks per month using this layered caching strategy.

"What happens if the database goes down?" This tests your understanding of availability and fault tolerance. Answer in layers: (1) The Redis cache continues serving redirects for cached URLs — depending on your cache hit ratio (typically 80-90%), most users will not notice. (2) For writes, you queue new URL creation requests in a message queue (Kafka or SQS) and process them when the database recovers. Return a "URL will be available shortly" response or a 503 with retry-after header. (3) For database durability, use synchronous replication to a standby instance that can be promoted to primary within seconds (PostgreSQL streaming replication with pg_auto_failover, or AWS RDS Multi-AZ). (4) If using DynamoDB or Cassandra, the database itself handles node failures — your data is replicated across multiple nodes and the system continues operating during single-node failures.

Common Mistakes in URL Shortener Design

These are the mistakes interviewers see repeatedly and the corrections that demonstrate depth.

Not considering the read-to-write ratio. URL shorteners are among the most read-heavy systems in existence. Bitly reports roughly 600 million link clicks per month versus about 6 million links created — a 100:1 ratio. Your architecture must optimize for reads above all else. This means aggressive caching (Redis cluster holding the top 20% of URLs), read replicas, and possibly a CDN for the most popular redirects. Candidates who focus on the write path and treat reads as an afterthought miss the fundamental characteristic of the problem.

Using auto-increment IDs as short codes. Sequential IDs like short.ly/1, short.ly/2, short.ly/3 are a security disaster: anyone can enumerate all URLs in your system by incrementing the ID. They also leak information about your traffic volume (if the latest URL is short.ly/50000000, you have created 50 million URLs). Use base62 encoding of the counter rather than the raw number, or better yet, use random short codes from a pre-generated pool.

Ignoring the cache layer. Without caching, every redirect hits the database. At 3,000+ reads per second, a single database server is under significant load. With a Redis cache and an 85% hit rate, only 450 reads per second reach the database — an 85% reduction. The cache key is the short code, the value is the long URL. Set a TTL based on URL popularity: popular URLs stay cached longer, rarely accessed URLs expire after an hour. Use cache-aside pattern: check cache first, on miss read from database and populate cache.

Not discussing analytics. Modern URL shorteners are as much analytics platforms as they are redirect services. Bitly's primary business model is analytics, not URL shortening. When you design the system, explain that each click generates an analytics event containing: short code, timestamp, IP address (for geolocation), user agent (for device and browser breakdown), and referrer (for traffic source analysis). These events are write-heavy and append-only, making them perfect for a time-series database (InfluxDB), event streaming platform (Kafka to S3 or BigQuery), or NoSQL store (Cassandra or DynamoDB). Separating the analytics write path from the redirect read path is critical — you never want analytics processing to slow down redirects.

Choosing the wrong redirect status code. HTTP 301 (Moved Permanently) tells browsers and search engines to cache the redirect and go directly to the long URL in future requests. This is faster for users but means you lose visibility into repeat clicks — your analytics undercount. HTTP 302 (Found / Temporary Redirect) tells browsers to check the short URL every time, which ensures every click is tracked but adds latency. Most URL shorteners use 302 for analytics accuracy, switching to 301 only for specific enterprise customers who prioritize performance over tracking. Discussing this tradeoff in an interview shows attention to real-world product requirements.

Original DosiBridge notes for Design URL Shortener

This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study Design URL Shortener is to connect the concept to a production service where growth exposes the first weak assumption in the original design. That framing changes the design discussion from definitions to operating choices.

What makes this topic distinct

• Core Entity: For Design URL Shortener, decide the core entity before choosing tools; otherwise the design looks clean but fails when url changes.
• Read/Write Path: For Design URL Shortener, decide the read/write path before choosing tools; otherwise the design looks clean but fails when shortener changes.
• Contention Point: For Design URL Shortener, decide the contention point before choosing tools; otherwise the design looks clean but fails when url changes.
• Abuse Path: For Design URL Shortener, decide the abuse path before choosing tools; otherwise the design looks clean but fails when shortener changes.
• Recovery Workflow: For Design URL Shortener, decide the recovery workflow before choosing tools; otherwise the design looks clean but fails when url changes.

Design URL Shortener correctness boundary

For this problem, separate the user-visible action from background work. The correctness boundary is the smallest state transition that must not be duplicated, lost, or reordered. Naming that boundary makes the solution more original than a stock API/database/cache template.

Design URL Shortener scaling bottleneck

The first bottleneck should be described in the language of this system: a hot timeline, a contested seat, a busy wallet ledger, a media-processing queue, or a search index update delay. That bottleneck decides which component deserves the deepest interview discussion.

Production review question

If Design URL Shortener suddenly receives 10x traffic, which entity becomes contested first, and what queue, lock, shard, or cache protects that entity without hiding correctness bugs?

Practical metric to track

Track url-shortener health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For Design URL Shortener, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.

Frequently Asked Questions About URL Shortener Design

What is the optimal length for a short code? Seven characters using base62 encoding (a-z, A-Z, 0-9) gives you 62^7 = approximately 3.5 trillion possible codes. At 100 million new URLs per year, this lasts over 35,000 years before exhaustion. Six characters gives 56 billion possibilities — still plenty for most use cases but might run low at extreme scale over decades. Shorter codes are better for usability (easier to type, share, and fit in SMS messages), but longer codes reduce collision probability when using hash-based generation. Bitly uses 7 characters, TinyURL uses 7-8, and Twitter's t.co uses 10 characters (they use a larger character set).

Should you use SQL or NoSQL for the URL mapping table? Both work well, but they optimize for different things. A relational database like PostgreSQL gives you ACID transactions (important if you need to guarantee that a short code is not double-assigned), mature tooling, and strong consistency on reads. It handles the URL shortener workload well up to tens of millions of URLs. A NoSQL key-value store like DynamoDB gives you automatic horizontal scaling, single-digit millisecond reads at any scale, and a natural fit for the access pattern (key-value lookup by short code). At Bitly's scale (billions of URLs), a distributed key-value store is the better fit. For an interview, start with PostgreSQL and explain when you would migrate to DynamoDB.

How do you handle URL expiration and cleanup? Support optional TTL on URL creation (the user specifies an expiration date). Store the expiration timestamp in the URL record. For redirect requests, check if the URL has expired before redirecting — return 410 Gone if expired. For cleanup, run a background job that scans for expired URLs and removes them from the database and cache. Use a database index on expiration timestamp for efficient scanning. For cache entries, set the Redis TTL to match the URL expiration so expired URLs are automatically evicted. TinyURL keeps URLs forever by default; Bitly allows enterprise customers to set expiration policies.

How do you prevent abuse of a URL shortening service? URL shorteners are frequently abused for phishing, malware distribution, and spam. Protection strategies: (1) Rate limiting — restrict URL creation to N per minute per IP address or authenticated user. Use a token bucket algorithm in Redis. (2) URL scanning — check submitted URLs against Google Safe Browsing API or similar blocklists before creating the short URL. (3) CAPTCHA for anonymous creation — require human verification for unauthenticated requests. (4) Link preview pages — instead of directly redirecting, show an interstitial page with the destination URL so users can verify before proceeding. (5) Monitoring and reporting — track click patterns and flag URLs with suspicious activity (sudden spike from unusual geographies, high click rate immediately after creation).

How would you implement custom aliases (vanity URLs)? Custom aliases like short.ly/summer-sale require additional validation: check that the alias is not already taken (uniqueness constraint), validate character restrictions (alphanumeric, hyphens, minimum length), and reserve certain aliases (api, admin, health) to prevent conflicts with system routes. Store custom aliases in the same table as generated short codes — they are functionally identical. The main engineering challenge is that custom aliases can conflict with future randomly generated codes, so reserve the custom alias namespace separately or check both tables during generation. Bitly charges a premium for custom aliases and limits them to paid accounts, which also reduces abuse.