1. How do you approach a system design interview?
Follow a structured framework: clarify requirements, estimate scale, sketch a high-level design, deep-dive into the riskiest components, then discuss bottlenecks and trade-offs. Start by asking about functional requirements, non-functional requirements (latency, availability, consistency) and expected traffic. Think aloud, state assumptions, and justify each choice by its trade-off rather than naming technologies. Reserve the last minutes for failure modes, monitoring, and future scaling.
1. Requirements (functional + non-functional)
2. Capacity estimation (QPS, storage, bandwidth)
3. API and data model
4. High-level diagram
5. Deep dive: DB, cache, queue, sharding
6. Bottlenecks, failures, trade-offs
2. What is the difference between vertical and horizontal scaling?
Vertical scaling (scale up) adds more CPU, RAM or disk to a single machine, while horizontal scaling (scale out) adds more machines and spreads load across them. Vertical is simple and needs no code changes, but it hits hardware limits and remains a single point of failure. Horizontal scaling gives near-unlimited capacity and fault tolerance, but requires load balancing, stateless services, and handling distributed data.
3. What is a load balancer and how does it work?
A load balancer distributes incoming requests across multiple backend servers to improve throughput, availability, and fault tolerance. It performs health checks, removes unhealthy servers, and routes traffic using an algorithm. It can operate at Layer 4 (TCP/UDP, faster) or Layer 7 (HTTP, able to route by URL, header or cookie). To avoid it being a single point of failure, run it in an active-passive or active-active pair.
4. What load balancing algorithms do you know, and when would you use each?
Common algorithms are round robin, weighted round robin, least connections, IP hash, and consistent hashing. Round robin suits similar servers with uniform requests. Least connections fits long-lived or uneven requests. Weighted variants handle servers of different capacity. IP hash or consistent hashing gives sticky routing, which is useful for caches or session affinity, and consistent hashing minimizes reshuffling when servers are added or removed.
5. What is caching and what are the main caching strategies?
Caching stores frequently accessed data in fast storage, such as memory, to cut latency and reduce load on the primary database. The main strategies are cache-aside (the app reads the cache, then the DB on a miss and populates the cache), read-through, write-through (write to cache and DB synchronously), write-back (write to cache first, flush to DB later), and write-around (write to DB only). Choose based on read/write ratio and how much staleness or data loss you can tolerate.
6. How do you handle cache invalidation and eviction?
Cache invalidation is handled by TTL expiry, explicit deletion or update on writes, or event-driven invalidation, while eviction policies decide what to remove when memory is full. Common policies are LRU (least recently used), LFU (least frequently used), and FIFO. Deleting the key on write is usually safer than updating it, because it avoids race conditions. TTLs with a little random jitter prevent many keys expiring at once.
7. What is a cache stampede and how do you prevent it?
A cache stampede (thundering herd) happens when a popular key expires and many concurrent requests all miss and hit the database at the same time. Prevent it with request coalescing or a per-key lock so only one request rebuilds the value, TTL jitter, serving stale data while refreshing in the background, and proactively refreshing hot keys before they expire.
8. What is a CDN and when should you use one?
A CDN (Content Delivery Network) is a geographically distributed network of edge servers that cache content close to users, reducing latency and origin load. Use it for static assets such as images, video, CSS, and JavaScript, and increasingly for cacheable API responses. Pull CDNs fetch content from the origin on the first request, while push CDNs have you upload content in advance. Control freshness with Cache-Control headers, versioned file names, or explicit purges.
9. How do you choose between SQL and NoSQL databases?
Choose SQL when you need structured data, complex joins, and strong ACID transactions, and choose NoSQL when you need flexible schemas, massive horizontal scale, or very high write throughput. NoSQL comes in several types: key-value (Redis, DynamoDB), document (MongoDB), wide-column (Cassandra), and graph (Neo4j). Start from the access patterns and consistency needs rather than hype; many systems use both, picking each store for the workload it fits.
10. What are ACID and BASE?
ACID describes relational transaction guarantees: Atomicity (all or nothing), Consistency (valid state transitions), Isolation (concurrent transactions do not interfere), and Durability (committed data survives crashes). BASE (Basically Available, Soft state, Eventual consistency) describes many distributed NoSQL systems that favor availability and scale over immediate consistency. ACID suits payments and inventory, while BASE suits feeds, counters, and analytics where brief staleness is acceptable.
11. What is database replication and what are its types?
Replication copies data across multiple database nodes to improve availability, read scalability, and durability. In leader-follower (primary-replica) replication, writes go to the leader and are copied to followers that serve reads. Multi-leader and leaderless setups accept writes on several nodes but need conflict resolution. Replication can be synchronous (stronger durability, higher write latency) or asynchronous (faster, but replicas may lag and lose recent writes on failover).
12. What is database sharding and how do you choose a shard key?
Sharding splits a large dataset horizontally across multiple database nodes so each holds only a subset of the rows, allowing writes and storage to scale beyond one machine. A good shard key has high cardinality, spreads load evenly, and matches common query patterns so most queries hit one shard. Common strategies are range-based, hash-based, and directory-based sharding. Drawbacks include hotspots, cross-shard joins and transactions, and difficult resharding.
13. What is consistent hashing and why is it used?
Consistent hashing maps both keys and nodes onto a hash ring, and each key is assigned to the next node clockwise, so adding or removing a node only moves a small fraction of keys instead of nearly all of them. It is used in distributed caches, sharded databases, and load balancers. Virtual nodes (many ring positions per physical node) even out the load and make rebalancing smoother.
14. What is the CAP theorem?
The CAP theorem states that during a network partition a distributed system must choose between consistency and availability; it cannot guarantee both. Consistency means every read sees the latest write, availability means every request gets a non-error response, and partition tolerance means the system keeps working despite dropped messages between nodes. Since partitions are unavoidable in real networks, the practical choice is CP (for example HBase, etcd) or AP (for example Cassandra, DynamoDB in default mode).
15. What is the difference between strong and eventual consistency?
Strong consistency guarantees that after a write completes, every subsequent read returns that write, while eventual consistency only guarantees that replicas converge to the same value if no new writes occur. Strong consistency costs extra latency and reduced availability, because nodes must coordinate. Eventual consistency is faster and more available, which suits social feeds and likes. Intermediate models include read-your-writes, monotonic reads, and causal consistency.
16. What is a message queue and why use one?
A message queue is a durable buffer that lets producers send messages that consumers process asynchronously, decoupling the two sides. It smooths traffic spikes, enables retries, and lets services scale and fail independently. Typical uses are email sending, image processing, and order pipelines. Examples include RabbitMQ, Amazon SQS, and Kafka. Design consumers to be idempotent, since most systems offer at-least-once delivery and duplicates can occur.
17. What is the difference between Kafka and a traditional message queue like RabbitMQ?
Kafka is a distributed, partitioned, append-only log where consumers track their own offsets and can replay retained messages, whereas RabbitMQ is a broker that pushes messages to consumers and typically removes them once acknowledged. Kafka excels at very high throughput, event streaming, and replay. RabbitMQ offers flexible routing, per-message acknowledgment, and priorities, which suits task queues and complex routing. Ordering in Kafka is guaranteed only within a partition.
18. What is the difference between REST, gRPC, and GraphQL, and how do you design a good API?
REST uses resources and HTTP verbs over JSON and is simple and cacheable, gRPC uses Protocol Buffers over HTTP/2 for fast, typed service-to-service calls, and GraphQL lets clients request exactly the fields they need from one endpoint. A good API is consistent, versioned, and paginated, uses proper status codes, and supports idempotency keys for unsafe retries. Prefer cursor-based pagination for large, changing datasets.
GET /v1/orders?limit=20&cursor=abc
POST /v1/orders (Idempotency-Key header)
GET /v1/orders/123
PATCH /v1/orders/123
DELETE /v1/orders/123
19. What is rate limiting and which algorithms can you use?
Rate limiting caps how many requests a client can make in a time window, protecting services from abuse, overload, and cost spikes. Common algorithms are token bucket (allows bursts, refills at a steady rate), leaky bucket (smooths output), fixed window counter (simple but allows boundary bursts), sliding window log, and sliding window counter. In distributed setups, keep counters in a shared store like Redis and return 429 Too Many Requests with a Retry-After header.
20. What is the difference between latency and throughput?
Latency is the time one request takes to complete, while throughput is how many requests or bytes the system processes per unit of time. They are related but not identical: batching or pipelining can raise throughput while increasing latency. Measure latency with percentiles such as p95 and p99 rather than averages, because tail latency is what users feel and it compounds across service calls.
21. What is high availability and how do you measure it?
High availability means a system stays operational and accessible for a very high percentage of time, usually expressed in nines. 99.9% allows about 8.8 hours of downtime per year, 99.99% about 53 minutes, and 99.999% about 5 minutes. Achieve it by removing single points of failure through redundancy, health checks, automatic failover, multi-zone or multi-region deployment, and graceful degradation. Availability of components in series multiplies, so dependencies reduce overall uptime.
22. How do you design for reliability and fault tolerance?
Assume every component will fail and design so the system keeps working anyway. Use redundancy, timeouts, retries with exponential backoff and jitter, circuit breakers, bulkheads to isolate failures, and idempotent operations so retries are safe. Add monitoring, alerting, and health checks, and test with chaos engineering and regular disaster recovery drills. Define RTO and RPO targets to decide how much redundancy and backup each service needs.
23. How do you do back-of-the-envelope estimation in a system design interview?
Estimate traffic, storage, and bandwidth with round numbers and simple arithmetic, stating your assumptions clearly. Start from daily active users, multiply by actions per user, and divide by about 100,000 seconds per day to get average QPS, then apply a 2x to 10x factor for peak. Multiply item size by volume and retention for storage. Precision is not the goal; the goal is to reveal whether you need sharding, caching, or a CDN.
100M DAU x 2 writes/day = 200M writes/day
200M / 100k s = ~2,000 writes/s avg, ~10k peak
500 bytes x 200M = 100 GB/day
100 GB x 365 x 5 yr = ~180 TB
24. How would you design a URL shortener like bit.ly?
Generate a unique short code for each long URL, store the mapping, and redirect on lookup. Use an ID generator (counter, snowflake ID, or pre-allocated key ranges) encoded in base62, giving 7 characters for about 3.5 trillion combinations, which avoids collision checks. Store mappings in a key-value or sharded SQL store, put a cache like Redis in front because reads vastly outnumber writes, and return 301 or 302 depending on whether you want analytics. Add expiry, rate limiting, and click-count queues.
25. How would you design a rate limiter as a distributed service?
Place the limiter at the API gateway or as middleware, identify clients by API key, user ID, or IP, and keep counters in a fast shared store such as Redis. Use an atomic operation, like a Lua script or INCR with EXPIRE, to avoid race conditions between servers. Token bucket is a common choice since it allows bursts. Make rules configurable, fail open or closed deliberately if Redis is down, and return 429 with Retry-After headers.
26. How would you design a chat application like WhatsApp?
Use persistent WebSocket connections to chat servers that route messages between users, with a message store for durability and offline delivery. A connection registry (often Redis) maps each user to the server holding their socket. Messages get a server-assigned sequence number per conversation for ordering, are stored before acknowledgment, and are pushed if the recipient is online or queued with a push notification if not. Use delivered and read receipts, and shard storage by conversation ID.
Sender -> Chat server A -> store message (conversation_id, seq)
-> lookup user->server in Redis
-> Chat server B -> WebSocket -> Recipient
offline: keep in inbox + send push notification
27. How would you design a news feed like Facebook or Twitter?
Generate each user's feed either at write time (fan-out on write) or at read time (fan-out on read), and usually combine both. Fan-out on write pushes new post IDs into followers' cached feed lists, making reads fast but expensive for accounts with millions of followers. Fan-out on read merges recent posts from followed accounts at request time. A hybrid pushes for normal users and pulls for celebrities, then ranks results and hydrates post details from a cache.
28. How would you design a notification system?
Accept events through an API, publish them to a message queue, and let separate workers deliver through channels such as push, email, SMS, and in-app. A central service checks user preferences, applies templates, deduplicates, and rate limits. Use per-channel queues so a slow SMS provider does not block push, retry with backoff, and track delivery status. Third-party providers such as APNs, FCM, or an email gateway sit at the edge.
29. How would you design a distributed cache like Redis or Memcached?
Partition keys across nodes with consistent hashing, keep data in memory with an eviction policy such as LRU, and replicate each shard for availability. Clients or a proxy route requests to the right node. Add replicas with automatic failover, optional persistence (snapshots or an append-only log), TTL support, and hot-key mitigation such as local caching or key splitting. Decide up front whether the cache can lose data, since most caches are allowed to.
30. How would you design a distributed unique ID generator?
Use a Snowflake-style 64-bit ID composed of a timestamp, a machine or datacenter ID, and a per-millisecond sequence number. This yields IDs that are unique without coordination, roughly time-sortable, and compact enough for database indexes. Random UUIDs are simpler but are 128 bits and poorly sortable, which hurts B-tree index locality. Beware of clock skew and clock rollback, and make sure each node gets a unique worker ID.
| 1 bit unused | 41 bits timestamp | 10 bits machine ID | 12 bits sequence |
4096 IDs per millisecond per machine
31. How would you design a file storage service like Dropbox or Google Drive?
Split files into fixed-size chunks, store the chunks in object storage, and keep file metadata (names, folders, chunk lists, versions) in a database. Hash each chunk so unchanged chunks are never re-uploaded, which enables deduplication and efficient delta sync. Clients upload directly to object storage using pre-signed URLs, and a notification service tells other devices to sync. Replicate chunks across zones and handle edit conflicts with versioning.
32. How would you design a video streaming service like YouTube or Netflix?
Upload the original video to object storage, then transcode it asynchronously into several resolutions and bitrates and split it into short segments. Serve segments over HTTP using adaptive bitrate protocols such as HLS or MPEG-DASH, so the player switches quality as bandwidth changes. Distribute segments through a CDN, keeping popular titles at the edge. Store metadata and recommendations separately, and use a queue plus worker fleet for transcoding.
33. How would you design search autocomplete (typeahead)?
Store popular query prefixes in a trie, or precomputed top-K suggestions per prefix, and serve them from memory with very low latency. Build the data offline from query logs, aggregating frequency counts periodically, and refresh it on a schedule. Cache responses at the CDN and browser, debounce keystrokes on the client, and shard the trie by prefix range. Filter offensive terms and optionally personalize results.
34. How would you design a ride-sharing service like Uber?
Track driver locations with frequent updates stored in a geospatial index, such as geohash, quadtree, or S2 cells, and query nearby drivers when a rider requests a trip. A matching service picks a driver based on distance, ETA, and availability, then a trip state machine tracks the ride. Drivers send location over WebSockets or MQTT, and writes go to an in-memory store because positions change constantly. Payments and pricing run as separate services.
35. How does a web crawler work and how would you design one?
A crawler starts from seed URLs, fetches pages, extracts links, and adds unseen links to a queue called the URL frontier. Key design points are politeness (respect robots.txt and per-domain rate limits), deduplication of URLs and content using hashes or Bloom filters, prioritization of important pages, and distributing work across many workers partitioned by domain. Handle traps like infinite URL loops, retries, and periodic recrawling of changing pages.
36. What is a Bloom filter and where is it used?
A Bloom filter is a space-efficient probabilistic data structure that tests whether an element is in a set, with possible false positives but never false negatives. It uses a bit array and several hash functions. It is used to avoid costly lookups, for example checking whether a key exists before querying a database or disk, in crawlers for seen URLs, and in LSM-tree stores like Cassandra. Standard Bloom filters do not support deletion.
37. What is leader election and how is it implemented?
Leader election is the process by which distributed nodes agree on a single node to coordinate work, such as accepting writes or scheduling jobs. It is typically implemented with a consensus protocol such as Raft or Paxos, or with a coordination service like ZooKeeper or etcd that grants a lease or ephemeral node. Leaders send heartbeats, and followers start a new election when they stop. Fencing tokens prevent a stale leader from corrupting data.
38. What are Paxos and Raft, and why do distributed systems need consensus?
Consensus protocols let a group of nodes agree on a value or an ordered log of operations even when some nodes or messages fail. Raft and Paxos both work by majority quorum, so they tolerate failure of a minority of nodes (for example, 2 of 5). Raft was designed to be easier to understand, with a clear leader, log replication, and term-based elections. It is used in etcd, Consul, and many databases for replicated state.
39. What is the difference between a quorum read and write, and how do R, W, and N work?
In a replicated store with N replicas, a write must be acknowledged by W nodes and a read must contact R nodes. If R + W > N, the read set and write set overlap, so a read sees at least one up-to-date replica, giving strong consistency in practice. Lowering W or R improves latency and availability but allows stale reads. Dynamo-style systems like Cassandra let you tune these per request.
N=3, W=2, R=2 -> R+W=4 > 3 -> reads see latest write
N=3, W=1, R=1 -> R+W=2 -> fast, but may be stale
40. What is idempotency and how do you implement idempotent APIs?
An operation is idempotent if performing it multiple times has the same effect as performing it once, which makes retries safe. For non-idempotent calls such as payments, the client sends a unique Idempotency-Key header, and the server stores the key with the result, returning the saved response for duplicates. Save the key atomically with the operation, for example using a unique database constraint, and expire old keys after a retention period. GET, PUT, and DELETE are idempotent by design.
41. What is the saga pattern and how does it handle distributed transactions?
A saga splits a distributed transaction into a sequence of local transactions across services, where each step has a compensating action that undoes it if a later step fails. This avoids slow, blocking two-phase commit across services. Sagas are coordinated by choreography (services react to events) or orchestration (a central coordinator directs the steps). They provide eventual consistency, so compensations must be idempotent and visible intermediate states must be tolerated.
42. What is the outbox pattern and what problem does it solve?
The outbox pattern guarantees that a database update and the publishing of a related event happen reliably together, avoiding the dual-write problem where one succeeds and the other fails. The service writes the business change and an event row to an outbox table in the same local transaction. A separate relay process, or change data capture (CDC) tool such as Debezium, reads the table and publishes events to the broker, giving at-least-once delivery.
43. What is event sourcing and CQRS?
Event sourcing stores every state change as an immutable event and derives current state by replaying events, giving a full audit trail and the ability to rebuild state. CQRS (Command Query Responsibility Segregation) separates the write model from one or more read models optimized for queries. They are often combined, with read models updated asynchronously from the event stream. The cost is complexity, eventual consistency, and the need for event versioning.
44. What are the three pillars of observability?
Observability rests on logs, metrics, and traces. Metrics are numeric time series, such as request rate, error rate, and latency percentiles, used for alerting and dashboards. Logs are detailed event records, best structured as JSON with a correlation ID. Distributed traces follow a single request across services, showing where time is spent. Define SLIs and SLOs on top, and alert on user-visible symptoms rather than every internal cause.
45. What is a circuit breaker and how does it differ from retries?
A circuit breaker stops calling a failing dependency after errors cross a threshold, failing fast instead of waiting for timeouts, and later lets a few trial requests through to test recovery. It has three states: closed, open, and half-open. Retries repeat a failed call hoping it succeeds, which can make an overloaded service worse through a retry storm. Use both together, with capped retries, exponential backoff with jitter, and a breaker to cut off sustained failure.
46. What is the difference between a monolith and microservices, and when should you not use microservices?
A monolith deploys all functionality as one unit, while microservices split it into independently deployable services with their own data. Microservices allow team autonomy and independent scaling, but add network latency, distributed transactions, operational overhead, and harder debugging. Avoid them for small teams, early-stage products, or unclear domain boundaries. A well-structured modular monolith is often the better starting point, splitting out services only when a real need appears.
47. What is an API gateway and how does it differ from a service mesh?
An API gateway is the entry point for external client traffic, handling routing, authentication, rate limiting, TLS termination, and request aggregation. A service mesh manages internal service-to-service traffic using sidecar proxies (for example Envoy in Istio or Linkerd), providing mutual TLS, retries, load balancing, and telemetry without changing application code. In short, the gateway handles north-south traffic, and the mesh handles east-west traffic.
48. How do you handle the hot key or hot partition problem?
A hot key is a single key or partition receiving disproportionate traffic, such as a celebrity profile, which overloads one node despite a balanced cluster. Mitigate it by caching the hot item in a local or replicated cache, splitting the key into several sub-keys with a random suffix and aggregating on read, adding read replicas, or isolating hot tenants. Detect hot keys with per-key metrics, and choose shard keys with enough cardinality to begin with.
49. How do you design for multi-region deployment and disaster recovery?
Deploy the system in multiple regions and route users with latency-based or geo DNS, choosing between active-passive (a standby region takes over on failure) and active-active (all regions serve traffic). Active-active needs cross-region data replication and conflict resolution, so it is more complex and often eventually consistent. Define RTO and RPO targets, automate failover, replicate backups across regions, and practice failover drills regularly rather than trusting an untested plan.
50. How do you handle large-scale data processing: batch versus stream processing?
Batch processing handles large bounded datasets on a schedule, offering high throughput and simple correctness, with tools such as Spark or MapReduce. Stream processing handles unbounded events continuously with low latency, using engines like Flink, Kafka Streams, or Spark Structured Streaming. Streams need handling for late events, windowing, and exactly-once or idempotent output. Many systems combine both, using streaming for fresh results and batch for recomputation and backfills.