Design Instagram

Problem Statement

Design a photo-sharing social network like Instagram. Users upload photos, follow other users, see a personalized feed of posts from people they follow, share temporary stories that expire in 24 hours, and discover content via an Explore page. The system handles 2B+ monthly users with 100M+ photos uploaded daily.

Requirements

Functional

• Upload photos with captions, filters, and location tags
• Generate a personalized feed ranked by relevance from followed accounts
• Stories: ephemeral posts that auto-delete after 24 hours
• Explore page: surface trending and personalized content from non-followed accounts

Non-Functional

• Latency: Feed loads in <500ms, photo upload confirmation in <3 seconds
• Availability: 99.99% for feed reads, eventual consistency acceptable (1-2 second delay for new posts)
• Scale: 2B monthly users, 500M DAU, 100M photos/day, 1M likes/second

Core Architecture

1. Photo Upload Pipeline -- Client uploads image to a pre-signed S3 URL. An async worker (triggered via SQS) generates multiple resolutions (thumbnail 150px, medium 640px, full 1080px), applies filters server-side if requested, strips EXIF data, and stores all variants in S3 with CDN distribution.

2. Feed Generation Service -- Hybrid push/pull model. For users with <500 followers (99% of users), new posts are fan-out-on-write: pushed to each follower's precomputed feed in Redis. For celebrity accounts (>500K followers), feed is fan-out-on-read: the follower's feed is assembled at read time by merging their precomputed feed with recent celebrity posts.

3. Stories Service -- Stories are stored with a TTL of 24 hours in a dedicated Cassandra table partitioned by user_id. A separate ring buffer per user holds story IDs. The stories tray (top of the feed) is precomputed for each user, ordered by recency and engagement probability.

4. Explore/Discovery Engine -- Kafka streams feed engagement events (likes, saves, shares) into a real-time trending detection pipeline. An embedding-based model matches user interests to content, combining collaborative filtering with image understanding (ResNet features). Candidate generation produces 10K posts, then a ranking model selects the top 50.

Database Choice

PostgreSQL for users, relationships (follows), and post metadata -- strong relational queries for follower graphs. Cassandra for the feed timeline and stories -- write-heavy, time-sorted, partitioned by user_id. S3 + CloudFront CDN for photo storage and delivery. Redis for precomputed feeds (sorted sets by timestamp) and user session cache. Elasticsearch for hashtag and user search.

Key API Endpoints

POST /api/v1/posts
  -> Body (multipart): \{ image: <file>, caption: "Sunset", location: "Bali", tags: ["travel"] \}
  -> Returns: \{ post_id: "P-456", image_urls: \{ thumb: "...", full: "..." \} \}

GET /api/v1/feed?cursor=\{timestamp\}&limit=20
  -> Returns: \{ posts: [...], next_cursor: "..." \}

GET /api/v1/stories/tray
  -> Returns: \{ trays: [\{ user_id: "U1", stories: [\{ id: "S1", media_url: "...", expires_at: "..." \}] \}] \}

Scaling Insight

The hybrid fan-out model is essential. Pure fan-out-on-write breaks for celebrities -- a single post by someone with 100M followers would generate 100M Redis writes. Pure fan-out-on-read is too slow for normal users. The hybrid approach: fan-out-on-write for 99% of users (fast reads, manageable writes), fan-out-on-read only for the top 0.1% celebrity accounts (merged at read time from a small set of celebrity timelines).

Key Tradeoffs

Deep-Dive: Clarifying Questions for Instagram

1. What is the posting volume? Instagram users share approximately 95 million photos and videos per day. With 2B+ monthly active users, the system handles massive read-heavy traffic.
2. How does the feed algorithm work? Instagram switched from chronological to algorithmic feed in 2016. The ranking considers: relationship (how often you interact with the poster), interest (predicted engagement based on content), timeliness, and session frequency.
3. What media processing is needed? Every uploaded photo needs: multiple resolution variants (thumbnail, standard, full), format conversion (HEIC to JPEG), face detection for tagging, content moderation (NSFW detection), and EXIF data stripping for privacy.
4. How does Explore work? The Explore page recommends content from accounts you do not follow, using collaborative filtering on engagement signals. It serves billions of impressions per day.
5. Do we need Stories and Reels? Stories (24-hour ephemeral content) and Reels (short-form video) have different storage, delivery, and ranking requirements than feed posts.
6. What about the social graph? The follow graph (who follows whom) is the foundation of feed generation and notifications.

Specific Functional Requirements

1. Photo/Video Upload: Upload images (up to 10 per post) and videos with filters, captions, hashtags, and location tags
2. Feed Generation: Algorithmic feed showing posts from followed accounts, ranked by predicted engagement
3. Explore Page: Discover content from accounts you do not follow, personalized by viewing and engagement history
4. Stories: Ephemeral photo/video content that disappears after 24 hours, with viewer lists
5. Direct Messages: Private messaging with text, images, and shared posts
6. Like, Comment, Share: Engagement features with real-time count updates
7. Search: Search by username, hashtag, and location

Specific API Endpoints

POST /api/v1/media/upload
  Body: multipart form (image files, caption, hashtags, location)
  Response: { "media_id": "abc123", "status": "processing", "url": "..." }

GET /api/v1/feed?cursor=xyz&limit=20
  Response: { "items": [{ "media_id": "...", "user": {...}, "images": { "thumbnail": "...", "standard": "...", "full": "..." }, "likes_count": 1234, "caption": "..." }], "next_cursor": "..." }

GET /api/v1/explore?cursor=xyz&limit=30
  Response: { "items": [...], "next_cursor": "..." }

POST /api/v1/media/:media_id/like
  Response: { "liked": true, "likes_count": 1235 }

GET /api/v1/users/:user_id/stories
  Response: { "stories": [{ "id": "...", "media_url": "...", "created_at": "...", "expires_at": "...", "viewers_count": 456 }] }

Specific Data Model

Media (PostgreSQL, sharded by user_id)

Feed Cache (Redis): user_id -> sorted set of media_ids with ranking scores. Pre-computed by feed ranking pipeline. Each user's feed cache holds ~500 ranked post IDs.

Social Graph (Cassandra/TAO): Optimized for "get followers of X" and "does A follow B" queries.

• Following: (user_id, followed_id, created_at)
• Followers: (user_id, follower_id, created_at)

Image Storage (S3): Each upload generates 4-5 variants stored in S3. Total photo storage: 95M uploads/day * 5 variants * 500KB average = ~240 TB/day of new image data.

Specific Back-of-the-Envelope Numbers

Traffic:

• 2B+ MAU, 500M+ DAU
• 95M photos/videos uploaded per day = ~1,100 uploads/second
• Feed reads: 500M users * 15 feed loads/day = 7.5B feed reads/day = ~87,000 reads/second
• Like actions: ~4 billion likes/day = ~46,000 likes/second

Storage:

• Photo storage: 95M * 5 variants * 500KB = 240 TB/day = 87 PB/year
• Database (metadata): each post ~1KB * 95M/day = 95 GB/day = 34 TB/year
• Feed cache (Redis): 500M users * 500 post IDs * 8 bytes = 2 TB

Image processing pipeline:

• 1,100 uploads/second, each generating 5 resized variants + content moderation
• Processing time per image: ~2 seconds
• Need ~5,500 concurrent image processing workers

CDN:

• Average user views ~50 images per session * 200KB per image = 10MB per session
• 500M sessions/day * 10MB = 5 PB/day of CDN bandwidth

Original DosiBridge notes for Design Instagram

This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study Design Instagram is to connect the concept to a fan-out system where freshness, delivery order, and hot users create the real scaling pressure. That framing changes the design discussion from definitions to operating choices.

What makes this topic distinct

• Core Entity: For Design Instagram, decide the core entity before choosing tools; otherwise the design looks clean but fails when instagram changes.
• Read/Write Path: For Design Instagram, decide the read/write path before choosing tools; otherwise the design looks clean but fails when instagram changes.
• Contention Point: For Design Instagram, decide the contention point before choosing tools; otherwise the design looks clean but fails when instagram changes.
• Abuse Path: For Design Instagram, decide the abuse path before choosing tools; otherwise the design looks clean but fails when instagram changes.
• Recovery Workflow: For Design Instagram, decide the recovery workflow before choosing tools; otherwise the design looks clean but fails when instagram changes.

Distinct angle: Instagram is media distribution plus ranking

Instagram-like design is driven by media ingest, transformations, feed ranking, and creator hot spots. The core tension is between immediate posting and delayed optimization: thumbnails, multiple resolutions, safety checks, captions, and recommendations all run at different speeds. A strong design separates the upload path from feed serving so failed media processing does not break basic social interactions.

Design Instagram correctness boundary

For this problem, separate the user-visible action from background work. The correctness boundary is the smallest state transition that must not be duplicated, lost, or reordered. Naming that boundary makes the solution more original than a stock API/database/cache template.

Design Instagram scaling bottleneck

The first bottleneck should be described in the language of this system: a hot timeline, a contested seat, a busy wallet ledger, a media-processing queue, or a search index update delay. That bottleneck decides which component deserves the deepest interview discussion.

Production review question

If Design Instagram suddenly receives 10x traffic, which entity becomes contested first, and what queue, lock, shard, or cache protects that entity without hiding correctness bugs?

Practical metric to track

Track instagram health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For Design Instagram, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.

Sources

• Design Instagram -- Reference
• Source: System-Design-Overview