Skip to main content
System Design Mafia
Back to patterns

Design Google Drive / Dropbox

A cloud-based file storage and sync service supporting multi-device uploads, downloads, sharing, and real-time sync with chunked uploads, CDN delivery, and delta sync.

synchronous: caller waits
asynchronous: fire & forget

Denser dots = higher throughput. Hover any edge for its full contract.

//The Contract

Requirements

Functionalwhat the system must do
  • ✓Users can upload files up to 50GB from any device with resumable chunked uploads and progress tracking
  • ✓Users can download files from any device via CDN-signed URLs for low-latency delivery
  • ✓Users can share files with other users and view files shared with them via ACL-based permissions
  • ✓Clients automatically sync file changes across devices using WebSocket push plus polling fallback
Non-functionalhow well it must do it
  • ≫High availability — prioritize availability over strict consistency; brief staleness is acceptable for file listings
  • ≫Low latency — CDN edge caching for downloads, chunked parallel uploads, and delta sync for file changes
  • ≫Reliability — files replicated across multiple zones; metadata backed up with operation logs
  • ≫Scalability — support millions of users, petabytes of storage, and thousands of concurrent uploads/downloads
//Back of the Envelope

Scale Estimates

Rough, order-of-magnitude numbers. The point is to justify the architecture, not to be exact.

Daily active users
100M
each syncing ~50 files per day across multiple devices
Storage volume
~10 PB
average 100GB per active user with deduplication reducing raw storage
Peak upload rate
50 GB/file
chunked into 5-10MB parts with multipart upload to object storage
Read : write ratio
10 : 1
downloads and sync checks dwarf uploads — CDN absorbs most read traffic
Sync latency target
< 5s
WebSocket push for active clients, polling fallback every few minutes
//Why It Looks Like This

System Evolution

No system is born with queues and caches. Every box was added because something was about to break. Play a stage and the diagram above shrinks to what existed at that point in the story.

01

The MVP — direct uploads to object storage

The simplest version: a client discovers the service via DNS, authenticates through an API Gateway and Auth Service, and uploads/downloads files through the File Service — which generates presigned URLs so file bytes flow directly to and from object storage.

Metadata (file name, size, fingerprint) is persisted in a DynamoDB table keyed by userId. This works for thousands of users, but every download comes from a single regional bucket — latency is high for global users and origin egress costs grow linearly.

02

Add the CDN for global downloads

Scale pressure: Users far from the origin region wait 200ms+ for every download, and origin egress costs dominate the infrastructure budget

A CDN now fronts object storage for downloads: DNS geo-routes to the nearest edge, and cache hits serve in <20ms. Signed URLs with short expiration (5 minutes) enforce access control at the edge.

The CDN also supports Range requests so interrupted downloads resume without restarting — critical for mobile networks. Popular files are cached globally; the long tail still streams from origin.

03

Add real-time sync across devices

Scale pressure: Users with multiple devices expect file edits on their laptop to appear on their phone within seconds — polling is too slow and too expensive at 100M DAU

A Sync Service maintains WebSocket connections per device for near-real-time push. When a file changes, the File Service publishes a file.changes event to a Kafka event bus. The Notification Worker consumes the event and pushes a sync notification through the WebSocket Manager to all of the user's online devices.

A Shared Files Cache (Redis) sits in front of the metadata DB to accelerate ACL lookups for frequently shared files. Offline devices get changes via periodic polling (GET /files/changes?since=...) when they reconnect.

04

Add sharing and email notifications

Scale pressure: File sharing is the primary growth vector — users need to share files and invite collaborators via email

The system reaches its full shape. An Email Provider sends share-invite notifications when a user shares a file with someone who doesn't yet have an account.

The Notification Worker also forwards events to the Email Provider asynchronously, keeping the sharing path decoupled from the notification delivery. This is the finished architecture: every tier — upload, download, sync, share — is decoupled by the event bus, and each scales independently.

//Follow the Data

Request Flows

A diagram shows what exists; a trace shows what happens. Press play on a flow, or click any step, and the diagram above lights up the exact path that request takes.

Flow 01

Upload a Large File

From selecting a file to it being available for download, with chunked resumable upload.

Flow 02

Download a Shared File

From requesting a file to receiving it via the nearest CDN edge.

Flow 03

Sync File Changes Across Devices

How a file edit on desktop propagates to mobile in near real-time.

//Component Breakdown

Key Components

Every box on the canvas, and the job it does. Click a card to locate it in the diagram; click the node itself for the full deep dive.

client×3
network×4
compute×4
database×1
cache×1
queue×1
storage×1
external×1
//Deep Dive

Architecture Breakdown

Architecture Overview

This design models a cloud file storage and sync service (Google Drive / Dropbox) built for scale, reliability, and low latency.

Key Design Principles

Separate control plane from data plane. File bytes never flow through our application servers. Clients upload and download directly to/from object storage (S3) and CDN edges using cryptographically signed URLs. Our backend (File Service, Sync Service) only handles metadata, permissions, and coordination — operations that are thousands of times lighter than moving multi-gigabyte files.

Chunked, resumable, deduplicated uploads. Files are split into content-defined chunks (CDC) on the client. Each chunk is fingerprinted (SHA-256) for global deduplication — if 1,000 users upload the same PDF, we store it once. Chunking also enables resumable uploads: a 50GB upload interrupted at 49GB only retries the remaining chunks, not the entire file.

Delta sync for fast synchronization. When a file changes, CDC ensures only the modified chunks are re-uploaded. A single-byte edit in a 1GB file might only transfer ~10MB. The sync service pushes changes in real-time via WebSocket and falls back to polling for reliability.

CDN-backed downloads with security. Downloads are served from 400+ CDN edge locations using signed URLs with short expiration (5 minutes). This minimizes latency for global users while preventing unauthorized link sharing. Range requests enable resumable downloads on flaky mobile networks.

Normalized permissions model. Sharing is implemented via a SharedFiles table (userId -> fileId) rather than embedded arrays. This turns 'find files shared with me' from an O(N) table scan into a single efficient query, with Redis caching hot lookups.

Fault Tolerance

  • Upload crashes: Metadata reconciliation jobs detect S3 objects that exist but metadata shows 'uploading', and repair the state idempotently.
  • Chunkserver / S3 zone failure: S3 provides 3x cross-zone replication by default.
  • WebSocket drops: Polling fallback guarantees eventual consistency.
  • CDN cache corruption: S3 ETags and checksums validate integrity on every origin fetch.
//Trade-offs

Design Decisions

An architecture is a record of trade-offs. For every major choice here: what won, what lost, and why the constraints made it so.

01

Direct client-to-storage upload vs proxy through backend

ChosePresigned URLs + multipart upload directly to S3
OverProxy uploads through File Service serversPeer-to-peer sync between clients

Video and document ingress is measured in petabytes. If file bytes flowed through our API servers, the compute tier would have to scale with storage traffic instead of request traffic — a 1000x difference. Presigned URLs let clients upload directly to S3 while our backend only handles lightweight metadata and URL generation. This pattern also enables resumable chunked uploads: if a 50GB upload fails at 49GB, only the remaining chunks need retrying.

02

Chunk boundary strategy for delta sync

ChoseContent-Defined Chunking (CDC) with Rabin fingerprinting
OverFixed-size chunks (e.g., 5MB blocks)Whole-file re-upload on any change

Fixed-size chunks shift all boundaries when a byte is inserted at the start, causing every subsequent chunk to get a new fingerprint and forcing a full re-upload. CDC uses a rolling hash to find chunk boundaries based on content, so a small edit only affects 2-3 surrounding chunks. This is how Dropbox achieves efficient delta sync in practice — a 1-byte edit in a 1GB file might only re-upload ~10MB instead of 1GB.

03

Real-time sync notification mechanism

ChoseWebSocket push with periodic polling fallback
OverPure polling every 30 secondsServer-Sent Events (SSE) onlyMQTT broker for all sync

WebSockets provide sub-second sync latency when devices are online, which is critical for the 'it just works' Dropbox experience. However, connections drop on mobile networks and sleep modes. A periodic poll every few minutes via GET /files/changes?since={timestamp} acts as a safety net to guarantee eventual consistency. We avoid SSE because we need bidirectional communication for the client to acknowledge sync state.

04

Sharing permission data model

ChoseNormalized SharedFiles table (userId -> fileId mapping)
OverEmbedded sharelist array in FileMetadataGraph database for complex ACLs

An embedded sharelist requires scanning every file's metadata to find files shared with a given user — O(N) where N is total files. A normalized SharedFiles table with userId as partition key and fileId as sort key turns this into a single efficient query. We keep a Redis cache in front for hot lookups. This trades a slightly more complex write path (updating two tables in a transaction) for a dramatically faster read path.

05

Download delivery optimization

ChoseCDN with signed URLs and Range request support
OverDirect S3 presigned URLs without CDNBackend proxy for all downloads

Users are globally distributed but S3 buckets are regional. A user in Sydney downloading from a US-East bucket faces 200ms+ latency per request. A CDN caches files at 400+ edge locations, reducing latency to <20ms for popular content. Signed URLs enforce that only authorized users can access cached content. Range requests let clients resume interrupted downloads without restarting — critical for mobile networks with intermittent connectivity.

//What Breaks First

Bottlenecks & Failure Modes

Reading a system means sensing where it cracks under 10× load. These are the pressure points of this design, and how it holds.

Risk 01

Viral file thundering herd

Failure mode

A shared folder with a popular file (e.g., a leaked document) receives millions of download requests simultaneously. The CDN origin (S3) and the File Service's metadata DB face a synchronized stampede.

Mitigation

The CDN absorbs 99%+ of download traffic via edge caching with request coalescing — concurrent identical cache misses collapse into a single origin fetch. The File Service caches ACL lookups in Redis with a 5-minute TTL. For truly viral events, we can temporarily increase the CDN cache TTL and serve slightly stale metadata.

Risk 02

Multipart upload metadata bloat

Failure mode

Millions of concurrent large file uploads create enormous metadata state in the FileMetadata DB (chunk arrays with hundreds of entries per file). DynamoDB item size limits (400KB) can be exceeded, and query performance degrades.

Mitigation

Store chunk manifests in a separate ChunkManifest table with fileId as partition key, or offload to S3 itself via the ListParts API. The FileMetadata table only tracks uploadId and overall status. For files with >1000 chunks, we paginate chunk status and verify in batches via S3 ListParts rather than tracking every chunk in the DB.

Risk 03

WebSocket connection exhaustion

Failure mode

With 100M DAU and an average of 2.5 devices per user, maintaining 250M persistent WebSocket connections overwhelms the Sync Service and underlying infrastructure.

Mitigation

Use a dedicated WebSocket gateway cluster (e.g., ElastiCache Redis Pub/Sub or a custom gateway) that maintains connections while the Sync Service remains stateless. Connections are sharded by userId across gateway instances. Idle connections are closed after 30 minutes; clients reconnect via polling until activity resumes.

Risk 04

Duplicate storage from deduplication gaps

Failure mode

Without global deduplication, the same file uploaded by thousands of users (e.g., a popular PDF, a standard template) is stored thousands of times, wasting petabytes of storage.

Mitigation

Fingerprint files with SHA-256 at upload time. If a file with the same fingerprint already exists, create a new metadata record pointing to the existing object rather than storing a new copy. Reference counting on objects ensures we only delete when the last reference is removed. This is how Dropbox's 'LanSync' and server-side deduplication work.

//Break Things on Purpose

Failure Lab

Most of this architecture exists because any one box can die. Take a component offline and the diagram above shows the blast radius: what goes down, what degrades, and what the user feels.

CDN (CloudFront)

CDN

Download latency jumps from ~20ms to 200ms+ as every request misses the cache and hits the origin S3 bucket directly. Origin egress costs explode.

File Service

REST API

Uploads and downloads fail — users cannot list, upload, or download files. Any files already cached at the CDN edge remain accessible via their signed URLs until they expire.

Sync Service

REST API

Real-time sync stops working — file changes no longer propagate instantly. The polling fallback (every few minutes) still catches changes, so no data is lost, only delayed.

Notification Worker

Worker

Real-time push notifications stop — newly uploaded files or edits on one device do not appear on others until they manually refresh or the polling interval fires.

File Metadata DB

DynamoDB

File listing, upload status checks, and share verification all fail. Files already cached at the CDN edge remain downloadable via existing signed URLs until they expire.

WebSocket Manager

Network

Real-time push notifications stop reaching active devices. The app appears to be in sync but silently misses updates until the user manually refreshes or the polling interval fires.

Object Storage (S3)

Object Storage

Uploads and downloads break entirely — new files cannot be stored and existing files cannot be served. The CDN edge cache continues serving files that were already cached, but any miss is fatal.

Event Bus (Kafka)

Kafka

Real-time sync and notifications stop — file changes still happen but are not propagated to other devices until the queue recovers. Uploads and downloads continue unaffected.

//Active Recall

Test Yourself

If you can answer these without scrolling back up, the architecture is yours. Try each one out loud before revealing.

//Go Deeper

Further Reading

Patterns used in this design
Design InstagramEvent-Driven Architecture
//A Word

From the Creator

Every great system starts as a sketch on a whiteboard. The ability to zoom out and see the whole picture (how services connect, where data flows, what breaks and why) is what separates engineers who build features from engineers who build systems. This diagram is more than boxes and arrows. It's a map of decisions, trade-offs, and intentional design.

Studying architectures isn't just about passing interviews. It's about training your intuition. The more systems you take apart, the better you get at sensing where a monolith will crack, where a queue belongs, or when a cache is hiding a deeper problem. You start seeing patterns instead of chaos.

So keep reading, keep tracing those edges, keep asking "why this way and not that way." The engineers who truly understand large systems are the ones who never stop being curious about how things fit together. That curiosity is the only ingredient that really matters.

Atharva Arbat@arbat_atharva

//Explore

More Patterns