System Design interview questions covering scalability, databases, caching, queues, APIs, reliability, consistency, and architecture tradeoffs.
Start by clarifying requirements before drawing architecture. Ask about users, core features, read/write patterns, latency goals, availability, scale, data retention, security, and constraints. A strong answer avoids designing a huge system before understanding what problem must be solved.
Functional requirements describe what the system does, such as upload photos or send messages. Non-functional requirements describe qualities such as latency, throughput, availability, durability, security, cost, and compliance. Both change design decisions.
Functional: create order, pay order, view order history
Non-functional: p95 latency < 300ms, 99.9% availability, audit logs retained 1 yearEstimate traffic, storage, bandwidth, and compute from assumptions. Example: 10 million daily active users, 5 requests per user per day, and 500 bytes per request gives 50 million requests/day and about 25 GB/day of request payload before overhead. Use estimates to size services and identify bottlenecks, not to pretend exactness.
QPS = daily_requests / 86,400
50,000,000 / 86,400 ~= 579 QPS
Peak QPS may be 3x to 10x averageAPI design should define resources, operations, request/response shape, authentication, pagination, error format, idempotency, and versioning. Keep APIs aligned with use cases, not database tables only.
POST /v1/orders
GET /v1/orders/{orderId}
GET /v1/users/{userId}/orders?cursor=abc&limit=20REST is simple and works well for resource-based public APIs. GraphQL helps clients fetch flexible shapes but needs careful authorization and query cost controls. gRPC is efficient for internal service-to-service communication with strongly typed contracts and streaming support.
Start from access patterns and consistency needs. Identify entities, relationships, primary keys, indexes, retention, and query patterns. A relational model is often best for transactions and joins, while NoSQL can fit high-scale key-value, document, or event workloads.
Read-heavy systems often benefit from caching, replicas, CDNs, and denormalized read models. Write-heavy systems need batching, partitioning, queues, backpressure, and careful database write capacity. The ratio helps decide where to optimize first.
Horizontal scaling adds more machines or instances instead of making one machine larger. It works best with stateless services because any instance can handle any request. Stateful components require partitioning, replication, or coordination.
Stateless services do not keep user session or workflow state in local memory. They are easier to scale, replace, deploy, and load balance. State should live in databases, caches, queues, or external stores.
Client -> Load Balancer -> API instance A/B/C
Session state -> Redis or database, not local process memoryA load balancer distributes traffic across healthy service instances. It can provide health checks, TLS termination, routing, sticky sessions, rate limits, and failover. Bad health checks or uneven routing can cause outages even when servers are running.
Replication copies data from one database node to others. It improves read scalability and availability, but replicas can lag. Systems that read from replicas must handle stale reads or route critical reads to the primary.
Sharding splits data across multiple database nodes by a shard key, such as user_id or tenant_id. It increases write and storage capacity but makes cross-shard queries, rebalancing, transactions, and operational work more complex.
shard_id = hash(user_id) % number_of_shardsChoose a key that distributes data evenly, matches common queries, avoids hot spots, and minimizes cross-shard operations. A poor shard key can overload one shard or force expensive fan-out queries.
Consistent hashing maps keys and nodes onto a ring so adding or removing nodes moves only a subset of keys. It is useful for caches, distributed storage, and sharded systems where rebalancing cost matters.
Caching stores frequently used data closer to the user or service, reducing latency and backend load. Caches can be browser caches, CDN caches, application caches, distributed caches, or database query caches. They add invalidation and consistency tradeoffs.
In cache-aside, the application checks the cache first. On a miss, it reads from the database, writes the result to cache with a TTL, and returns it. On updates, it invalidates or refreshes the cache.
value = cache.get(key)
if value is None:
value = database.read(key)
cache.set(key, value, ttl=300)Cache invalidation removes or updates stale cached data. Strategies include TTLs, explicit delete on write, versioned keys, event-driven invalidation, and write-through caching. The risk is serving stale data or causing cache stampedes.
A cache stampede happens when many requests miss the same key at once and all hit the database. Mitigations include locks, request coalescing, TTL jitter, stale-while-revalidate, and prewarming important keys.
A Content Delivery Network caches static or semi-static content near users. It reduces latency, bandwidth cost, and origin load. CDNs are common for images, videos, downloads, scripts, styles, and public API responses that can be cached safely.
Object storage stores large blobs such as images, videos, backups, exports, and documents. Instead of putting large files in a database, store metadata in the database and file content in object storage.
Database: file_id, owner_id, object_key, content_type, size
Object storage: uploads/2026/06/file.pngQueues decouple producers from consumers, absorb traffic spikes, enable retries, and move slow work out of request paths. They are useful for emails, payments, video processing, notifications, indexing, and integration workflows.
API -> Queue -> Worker -> Email provider
API returns quickly while worker handles slow email sendingEvent-driven architecture publishes events when facts occur, such as OrderPaid or UserRegistered. Consumers react asynchronously. It improves decoupling and scalability, but introduces eventual consistency, ordering, replay, and idempotency challenges.
Idempotency means repeating the same operation has the same effect as running it once. It is essential for retries, payments, order creation, and message processing. Use idempotency keys or unique constraints to prevent duplicate side effects.
POST /payments
Header: Idempotency-Key: pay_123
If key already processed, return original result instead of charging again.Rate limiting restricts requests by user, IP, API key, tenant, or endpoint. Common algorithms include fixed window, sliding window, token bucket, and leaky bucket. The design must define limits, burst behavior, storage, and error responses.
Backpressure slows or rejects incoming work when downstream systems cannot keep up. It protects databases, queues, and services from collapse. Techniques include bounded queues, rate limits, circuit breakers, load shedding, and retry budgets.
A circuit breaker stops calling a failing dependency after errors cross a threshold. It moves from closed to open, then half-open to test recovery. It prevents cascading failure and gives dependencies time to recover.
closed: send requests normally
open: fail fast
half-open: allow a few test requestsEventual consistency means replicas or derived views may be temporarily stale but converge later. It is common in distributed systems, caches, search indexes, and async workflows. Product behavior must make stale states acceptable or visible.
Strong consistency means reads observe the latest successful write according to the system contract. It simplifies user expectations but may increase latency, reduce availability during partitions, or require coordination across nodes.
Choose SQL for relational data, joins, constraints, transactions, and flexible querying. Choose NoSQL when access patterns are simple but scale, availability, or schema flexibility dominate. Many real systems use both for different parts.
Search indexing creates searchable representations of data in systems like Elasticsearch, OpenSearch, Solr, or database full-text indexes. Search is often eventually consistent because documents are indexed asynchronously after source data changes.
Real-time updates can use WebSockets, Server-Sent Events, push notifications, or polling. Choose based on directionality, scale, firewall compatibility, latency needs, and client type. WebSockets fit bidirectional communication; SSE fits server-to-client streams.
A multi-region design runs services in more than one geographic region for latency, availability, or disaster recovery. Key decisions include active-active vs active-passive, data replication, conflict resolution, routing, failover, and compliance.
Failover moves traffic or leadership from a failed component to a healthy one. It can be automatic or manual. Good failover design includes health checks, data replication, runbooks, testing, and clear rollback paths.
RPO, Recovery Point Objective, is how much data loss is acceptable. RTO, Recovery Time Objective, is how long recovery may take. These numbers drive backup frequency, replication design, standby capacity, and disaster recovery cost.
RPO = 5 minutes means at most 5 minutes of data loss
RTO = 30 minutes means service should recover within 30 minutesSLI is the measured signal, such as successful request latency. SLO is the internal target, such as 99.9% of requests under 300 ms. SLA is the external commitment, often with contractual consequences.
Observability is the ability to understand system behavior from outputs such as metrics, logs, traces, events, and profiles. A well-designed system exposes enough signals to diagnose failures without guessing.
Monitor request rate, error rate, latency, saturation, queue depth, cache hit rate, database connections, replication lag, CPU, memory, disk, dependency health, and business KPIs. Tie alerts to user impact, not every noisy internal metric.
Logs capture discrete events and context. Metrics are numeric time-series signals. Traces follow a request across services. Together they help answer what happened, how often it happens, and where time was spent.
Distributed tracing follows a request through multiple services using trace IDs and spans. It helps identify slow dependencies, fan-out patterns, retries, and service boundaries. It is especially useful in microservices.
Authentication verifies identity; authorization controls access. Design choices include sessions, JWTs, OAuth, API keys, RBAC, ABAC, tenant isolation, token expiration, revocation, and audit logging.
Use pre-signed object storage uploads, store metadata in a database, process files asynchronously, scan for malware, generate thumbnails in workers, and serve files through CDN. Avoid proxying large files through app servers when possible.
Client -> API asks for signed upload URL
Client -> Object Storage direct upload
Storage event -> Queue -> Worker processes fileCore requirements include creating short codes, redirecting quickly, preventing collisions, analytics, expiration, abuse controls, and high read traffic. A simple design stores code to long URL mapping in a key-value store and caches popular redirects.
POST /urls -> { code: "aB91x" }
GET /aB91x -> 302 Location: https://example.com/long-urlA notification system needs templates, user preferences, channel routing, retries, deduplication, rate limits, provider failover, and delivery tracking. Use queues so slow providers do not block user requests.
A chat system needs message storage, conversation membership, real-time delivery, offline sync, unread counts, push notifications, ordering, and abuse controls. WebSockets can handle online delivery, while persistent storage and queues handle offline and retry behavior.
A news feed can use fanout-on-write, fanout-on-read, or a hybrid approach. Fanout-on-write precomputes feeds for fast reads but can be expensive for celebrities. Fanout-on-read computes on demand but can be slower. Hybrid designs treat high-fanout users differently.
Payment systems need idempotency, audit logs, secure token handling, state machines, retries, provider webhooks, reconciliation, fraud checks, and strong consistency around money-moving records. Never rely only on client-side payment status.
Leaderboards need ranking, score updates, top-N queries, nearby-rank queries, tie handling, and periodic resets. Redis sorted sets are a common fit for real-time rankings, while durable storage keeps historical records.
ZADD leaderboard 1200 user:42
ZREVRANGE leaderboard 0 9 WITHSCORES
ZREVRANK leaderboard user:42Common mistakes include skipping requirements, overengineering too early, ignoring data consistency, forgetting failure modes, missing observability, assuming infinite cache, picking trendy tools without tradeoffs, and not explaining bottlenecks or alternatives.
State the decision, why it fits the requirements, what it costs, and when you would change it. Example: choose cache-aside for simple read scaling, but mention stale data, invalidation, stampede prevention, and metrics needed to validate it.
End by reviewing requirements, bottlenecks, scaling path, failure modes, data consistency, security, observability, cost, and rollout plan. This shows you can operate the design, not just draw boxes.
Checklist:
- Requirements met?
- Bottlenecks identified?
- Failure modes handled?
- Data consistency clear?
- Metrics/logs/traces planned?
- Security and cost addressed?Explore 500+ free tutorials across 20+ languages and frameworks.