Question 1

What are the common Redis caching patterns?

Accepted Answer

Redis caching patterns define how your application coordinates reads and writes between the cache and the source-of-truth database. Choosing the wrong pattern causes stale data, cache stampedes, or write amplification.

**Cache-aside (lazy loading)** is the most common pattern. The application checks Redis first; on a miss it fetches from the DB, writes the result to Redis with a TTL, and returns it. Reads that hit cold keys pay a latency penalty, but the cache only holds data that was actually requested. You own the invalidation logic, which makes it flexible but error-prone.

**Read-through** moves that miss-fill logic into a cache client library or sidecar. The application always talks to the cache; the cache fetches from the DB on misses transparently. This simplifies application code but requires a caching layer that understands your data model. It still suffers the cold-start penalty on first access.

**Write-through** writes to the cache and the database synchronously on every mutation. Reads are always fast because the cache is never stale. The cost is write latency (two writes per mutation) and wasted cache space for data that is written but never read.

**Write-behind (write-back)** writes to Redis immediately and asynchronously flushes to the database. This gives sub-millisecond write latency and can batch DB writes for efficiency. The risk is data loss if Redis crashes before the flush completes — acceptable for metrics or analytics, dangerous for financial records.

Two operational concerns apply to all patterns. **Cache stampede** happens when many requests simultaneously miss the same key (e.g., after TTL expiry). Mitigate with probabilistic early expiration, a mutex lock (Redis SET NX), or request coalescing. **Cache warming** matters at startup — a cold cache under full traffic can overwhelm the database. Pre-warm from a dump or route a fraction of traffic to the cache before full cutover.

In practice most systems combine patterns: cache-aside for reads, write-through for critical writes, and write-behind for high-frequency counters or event streams.

Question 2

What is the difference between vertical and horizontal scaling?

Accepted Answer

Scaling is how you increase a system's capacity to handle more load. The two fundamental approaches differ in *where* you add resources.

**Vertical scaling** (scaling up) means adding more resources to an existing machine — more CPU cores, more RAM, faster disks, or a bigger network card. It is operationally simple: you upgrade the hardware or VM size, and the application often needs no changes. However, it has a hard ceiling — there is a maximum machine size, and at the high end, the cost grows super-linearly (a machine with 4x the RAM often costs more than 4x). Crucially, a single larger machine is still a single point of failure, so vertical scaling does not improve availability.

**Horizontal scaling** (scaling out) means adding more machines and distributing load across them. This is the foundation of cloud-native architecture. It provides effectively unlimited capacity (add nodes as needed), and multiple nodes mean that a single node failure does not take down the service — improving fault tolerance. The trade-offs are complexity: stateless services scale horizontally almost trivially, but stateful services (databases, session stores) require careful partitioning (sharding), consistent hashing, or consensus protocols.

For stateless application tiers (web servers, API servers), horizontal scaling is the default choice. Auto-scaling groups or Kubernetes HPA can add and remove instances based on CPU or request rate, making it cost-efficient and resilient. For databases, the traditional path is to scale vertically first (because it requires no application changes), then scale out reads with replicas, and finally shard writes horizontally only when necessary — sharding adds significant complexity to queries and transactions.

In practice, most large systems use a combination. The database primary is sized vertically to its practical maximum, reads are distributed across horizontal replicas, and the stateless application layer scales horizontally with auto-scaling. The choice at any layer depends on the marginal cost of the next vertical upgrade vs. the operational cost of adding another node.

The key insight is that horizontal scaling requires the application to be designed for it — shared-nothing architecture, externalized session state, idempotent operations, and distributed tracing become necessary.

Question 3

What are the common approaches to API versioning?

Accepted Answer

API versioning lets you evolve a public API without breaking existing clients. The four main approaches each make different trade-offs between discoverability, client coupling, and REST purity.

**URI path versioning** (`/v1/users`, `/v2/users`) is the most common approach. The version is explicit, visible in logs and browser address bars, easy to route at the load balancer level, and simple to document. The downside is that it violates REST's resource identity principle — `/v1/users/123` and `/v2/users/123` represent the same resource. It also encourages wholesale version duplication rather than incremental evolution.

**Query parameter versioning** (`/users?version=2`) is easy to implement and keeps the base URL stable, but query params are semantically for filtering, not versioning. Caching behavior is complicated — many CDNs and proxies treat query strings differently, and some strip or ignore them.

**Header versioning** (`Accept-Version: 2` or a custom `X-API-Version` header) keeps URLs clean and aligns with HTTP's content negotiation model. It is harder to test in a browser (can't just type the URL) and less discoverable. This is the approach used by Stripe (`Stripe-Version: 2023-10-16`) and GitHub.

**Content negotiation / Accept header** (`Accept: application/vnd.myapi.v2+json`) is the most RESTful approach — the same URL returns different representations based on the Accept header. Used by GitHub's API. It is the least discoverable and hardest for API consumers unfamiliar with HTTP media types.

In practice, URI versioning is the pragmatic winner for public APIs because it is immediately understandable and easily tested with curl. Header versioning is preferred by sophisticated API platforms (Stripe) because it ties versions to timestamps (breaking changes only on specific dates) rather than requiring parallel codebases.

A critical strategy regardless of versioning scheme is **additive-only changes without version bumps**: adding optional fields, new endpoints, and new enum values are non-breaking. Breaking changes — removing fields, changing types, altering semantics — should require a version bump. The goal is to minimize the number of major versions that must be maintained in parallel.

Version sunset policy (deprecation warnings via `Deprecation` and `Sunset` headers) and a clear end-of-life timeline are as important as the versioning scheme itself.

Question 4

What is the difference between SLOs and SLAs?

Accepted Answer

SLOs, SLAs, and SLIs form a hierarchy for defining, measuring, and committing to reliability. Understanding how they relate is central to site reliability engineering.

An **SLI (Service Level Indicator)** is the raw measurement — the actual metric that tells you how the system is behaving. Examples: request success rate (successful requests / total requests), request latency at the 99th percentile, data freshness, availability uptime. An SLI must be precisely defined: which requests count as 'successful'? 5xx errors only, or also timeouts? This precision matters because SLOs are built on SLIs.

An **SLO (Service Level Objective)** is an internal target: a threshold on an SLI that the engineering team commits to meeting. For example: "99.9% of HTTP requests will succeed over a 30-day rolling window" or "p99 latency will be below 200ms." SLOs are owned by engineering. They are not disclosed to customers — they are the operational targets that, if met, give confidence the SLA will not be breached. SLOs should be set *tighter* than SLAs to leave a safety margin.

An **SLA (Service Level Agreement)** is a contract between a service provider and a customer. It typically specifies both the reliability commitment (often weaker than the internal SLO — e.g., 99.5% uptime) and the consequences of breach (service credits, refunds, contract penalties). SLAs are negotiated by business and legal teams, not engineering alone. Breaching an SLA has financial and reputational consequences.

The critical operational concept built on SLOs is the **error budget**: if the SLO is 99.9% availability, the error budget is 0.1% of request failures — equivalent to ~43 minutes of downtime per 30-day window. The error budget is consumed by outages, slow deployments, and risky changes. When the error budget is depleted, engineering must stop feature work and focus on reliability improvements. When the budget is healthy, engineering can move faster and take more risk. This creates an objective, data-driven framework for balancing reliability investment against feature velocity.

A common failure mode is setting SLOs aspirationally rather than empirically — start by measuring your current SLI baseline and set the initial SLO slightly below it, then tighten as reliability improves.

Question 5

What does 'five 9s' availability mean and how do you achieve it?

Accepted Answer

Five 9s refers to 99.999% availability, which translates to approximately 5 minutes and 15 seconds of total downtime per year. To put this in perspective, 99.9% (three 9s) allows roughly 8.7 hours per year, and 99.99% (four 9s) allows about 52 minutes. Each additional 9 represents a tenfold reduction in allowable downtime and a roughly tenfold increase in effort and cost.

Achieving five 9s requires eliminating single points of failure across every layer — compute, storage, networking, DNS, load balancing, and deployment. Every component must be redundant, and the system must automatically detect failures and failover without human intervention, because humans cannot respond within a 26-second monthly budget.

Redundancy is the foundation: multiple instances across availability zones or regions, replicated databases with automatic promotion, redundant network paths. Active-active architectures are preferred over active-passive because they avoid failover latency.

Automated failover and self-healing are mandatory. Health checks must detect failures within seconds. Circuit breakers prevent cascading failures. The system must degrade gracefully.

Deployment practices directly impact availability. Blue-green or canary deployments limit blast radius. Feature flags enable instant rollback. Zero-downtime database migrations are essential.

The cost-complexity tradeoff is the most important practical consideration. Moving from four to five 9s might mean multi-region active-active deployment, globally distributed databases, chaos engineering programs, and 24/7 on-call with under-5-minute response SLAs. Infrastructure cost can increase 10-100x.

Availability is also composed across dependencies. If your system depends on three services each at 99.99%, the composite availability is at most 99.97%. This is why organizations define error budgets per service.

Very few systems truly need five 9s. Most applications are well-served by three to four 9s.

Question 6

What is the difference between latency and throughput?

Accepted Answer

Latency and throughput are the two primary axes of performance, and they measure fundamentally different things.

**Latency** is the time elapsed between a request being sent and its response being received — the 'delay' experienced by a single operation. It is typically measured in milliseconds (or microseconds for high-performance systems) and reported at percentiles: p50 (median), p95, p99, p99.9. The tail latency (p99, p99.9) is often more operationally important than the median because it represents the worst-case experience for a real fraction of users. A system can have a p50 latency of 10ms and a p99 latency of 2 seconds, masking severe tail behavior.

**Throughput** is the amount of work a system can complete in a unit of time — typically measured in requests per second (RPS), transactions per second (TPS), or data transfer rate (MB/s). Throughput describes the system's capacity ceiling and is the primary metric for batch processing, data pipelines, and high-volume APIs.

The relationship between latency and throughput is described by **Little's Law**: `L = λW`, where L is the average number of requests in the system, λ is the arrival rate (throughput), and W is the average time in system (latency). As throughput approaches a system's capacity, queuing delays increase latency sharply — this is the fundamental trade-off. At low load, latency is dominated by processing time. At high load, it is dominated by queue wait time.

In practice, they often trade off: batching operations improves throughput by amortizing per-request overhead but increases latency for individual requests. Conversely, parallelism can reduce latency for individual complex queries but may reduce overall throughput by increasing resource contention.

Optimization strategies differ: improving latency requires reducing coordination overhead, co-locating data with computation, and minimizing lock contention. Improving throughput requires maximizing parallelism, minimizing I/O stalls, and efficient use of CPU pipelines and memory bandwidth.

Good system design establishes separate SLOs for latency (p99 < 200ms) and throughput (system handles 10,000 RPS) — they require different optimization approaches and measurement strategies.

Question 7

What are LRU, LFU, and FIFO cache eviction strategies?

Accepted Answer

Cache eviction policies determine which item is removed when the cache is full and a new item must be inserted. The right policy depends on the access pattern of the workload.

**FIFO (First In, First Out)** evicts the oldest item by insertion order, regardless of access frequency or recency. It is the simplest to implement (a queue) but performs poorly for workloads with a stable working set — an item inserted long ago but accessed frequently will be evicted simply because it was inserted first.

**LRU (Least Recently Used)** evicts the item that was accessed least recently. The intuition is temporal locality: if you used something recently, you are likely to use it again soon. LRU is the most commonly deployed policy (Redis default, CPU hardware caches) because it works well for workloads with a temporal locality pattern. The standard implementation uses a doubly-linked list combined with a hash map, giving O(1) get and put. The limitation is that a single sequential scan (e.g., a large batch job reading unique keys) can flush the entire cache — an effect called **cache pollution**.

**LFU (Least Frequently Used)** evicts the item accessed the fewest times. This suits stable, long-lived working sets where some items are permanently hot (home page, popular product). A naive implementation requires counting accesses per key and finding the minimum — O(log n) with a min-heap. An O(1) implementation exists using two hash maps and a doubly-linked list of frequency buckets (Aho et al.). The weakness of LFU is the **aging problem**: an item accessed 1000 times last week but never since will have a very high count and resist eviction even when it is no longer relevant. LFU-with-decay (dividing counts periodically) or Window-TinyLFU (used by Caffeine, the Java cache, and adopted by Redis's LFU mode) addresses this with a sketch-based frequency estimator plus a recency window.

**LRU vs. LFU in practice**: use LRU for general-purpose caches with temporal locality. Use LFU (or W-TinyLFU) when the access distribution is highly skewed and the hot set is stable — content delivery caches, DNS caches, read-heavy database query caches. Redis supports both (`allkeys-lru` and `allkeys-lfu` maxmemory policies). Memcached uses a segmented LRU variant.

Question 8

What is the difference between a stateless and stateful web tier?

Accepted Answer

**Stateful web tier**: each web server stores session data locally — in memory or on local disk. A user who authenticates on server A must always be routed back to server A because only that server holds their session. This is enforced via sticky sessions (load balancer affinity). Problems: if server A crashes, the session is lost; scaling horizontally requires either migrating sessions or dropping them; deployments that replace servers terminate sessions.

**Stateless web tier**: no session data lives on the web server. Every request carries all the context needed to process it — either in a token (JWT carries identity and claims) or by looking up shared state in an external store (Redis, Memcached, database). Any web server can handle any request from any user at any time.

**Why stateless is preferred at scale:**
- Load balancers can distribute requests to any healthy instance — no sticky sessions
- Auto-scaling is clean: add/remove instances without session migration
- Rolling deployments work without session drainage delays
- Simpler failover — crashed server is replaced transparently

**JWT approach**: the client stores a signed token. The server validates the signature on every request with no lookup. Tradeoff: tokens cannot be invalidated before expiry without a blocklist (which reintroduces shared state). Use short expiry + refresh token rotation.

**Shared session store approach**: sessions live in Redis. Web servers are stateless; Redis holds state. This allows immediate invalidation and richer session data. Redis becomes a stateful tier — it must be highly available with replication.

**In practice**, most production systems are stateless web tier + stateful data tier. The database and cache are inherently stateful; the compute layer is kept stateless. This separation is what allows horizontal scaling of web servers while maintaining data consistency.

The system design interview framing: when asked to scale a web app, moving from stateful to stateless is usually one of the first architectural changes to propose.

Question 9

What are the challenges of a multi-data center architecture?

Accepted Answer

Running across multiple data centers (or regions) eliminates single points of geographic failure and reduces latency by serving users from the nearest location. However, it introduces a set of non-trivial challenges:

**Data consistency and replication lag**: writes to one data center must propagate to others. During the replication window, reads from a secondary region may return stale data. Systems must choose a consistency model: strong consistency (synchronous replication — high latency), eventual consistency (async replication — stale reads possible), or causal consistency (partial ordering guarantees).

**Traffic routing and DNS failover**: GeoDNS routes users to the nearest healthy data center. DNS TTLs are typically 60–300 seconds, so failover is not instantaneous. Applications must handle the window between a data center going down and DNS propagating the change — connection pooling, retries, and circuit breakers reduce user impact.

**Failover automation**: detecting a failed data center reliably is harder than it sounds. Split-brain scenarios occur when a data center is isolated (network partition) but not down — two regions both believe they are primary and accept writes. Solving this requires a quorum-based consensus mechanism (Raft, Paxos) or explicit tie-breaking logic.

**Data residency and compliance**: GDPR and similar regulations may prohibit replicating EU user data to US data centers. This creates partitioned data tiers where not all data can be multi-region.

**Deployment complexity**: deploying a new version across regions requires coordination — deploy to one region, validate, then proceed. Schema migrations must be backward-compatible with both old and new application versions simultaneously (expand/contract pattern).

**Cost**: cross-region data transfer is expensive. Minimizing cross-region traffic through caching, read replicas, and careful routing is essential.

The canonical pattern: active-active for stateless web/cache tiers, active-passive or quorum-based for the write path to the primary database. Global load balancing (Route 53 health checks, Cloudflare) handles traffic steering.

Question 10

How do you estimate QPS in a back-of-envelope calculation?

Accepted Answer

Back-of-envelope QPS estimation follows a top-down decomposition:

**Step 1: Establish user volume**
Start with total users, derive DAU. For a social app with 300M users, assume 50% DAU = 150M daily active users.

**Step 2: Estimate requests per user per day**
Be specific about the action. A Twitter-like feed has: 3 timeline refreshes/day (reads) + 0.1 tweets/day (writes). That gives 150M × 3 = 450M read requests/day and 150M × 0.1 = 15M write requests/day.

**Step 3: Convert to per-second**
86,400 seconds in a day. Round to 10^5 for easier math.
450M / 100K ≈ 4,500 read QPS average
15M / 100K ≈ 150 write QPS average

**Step 4: Apply peak multiplier**
Average QPS understates real load. Traffic peaks — typically 2–3× average for consumer apps, up to 10× for events (sports, flash sales). Use 3× as a default: peak read QPS ≈ 13,500, peak write QPS ≈ 450.

**Step 5: State the read/write ratio**
In this case 30:1. This matters for architecture: read-heavy systems use read replicas and caches; write-heavy systems need write sharding or queue-based buffering.

**Key numbers to know:**
- 86,400 ≈ 10^5 seconds/day
- 1M requests/day ≈ 12 QPS
- 1B requests/day ≈ 12,000 QPS

**What interviewers look for**: not the exact number, but structured decomposition, stated assumptions, and knowing whether the answer implies a single server (< 1K QPS), a small cluster (1–10K QPS), or distributed infrastructure (> 100K QPS).

Question 11

How do you estimate storage needs in a back-of-envelope calculation?

Accepted Answer

Storage estimation follows a data-type decomposition:

**Step 1: Identify what gets stored**
Break the system into data types: user records, messages, images, videos, metadata. Each has a different size profile.

**Step 2: Estimate size per object**
Use standard benchmarks:
- Text record (user profile, tweet): 1 KB
- Image (compressed, thumbnail): 100–300 KB; full-size: 1–5 MB
- Video (1 min, compressed): 50–100 MB; 1 hr HD: 1–2 GB
- Audio (1 min MP3): 1 MB

**Step 3: Estimate volume per day**
For a Twitter-like system with 150M DAU and 0.1 tweets/user/day:
- 15M tweets/day × 1 KB = 15 GB/day text
- 30% include an image (4.5M images/day × 200 KB = 900 GB/day)

**Step 4: Project over the design horizon**
Interviewers typically want 5 years. 5 years ≈ 1,800 days.
- Text: 15 GB/day × 1,800 = 27 TB
- Images: 900 GB/day × 1,800 = 1.6 PB

**Step 5: Add replication overhead**
Data is typically stored with 3× replication (HDFS default, S3 default). Multiply total by 3.

**Step 6: State which tier**
- Hot storage (SSD): recent data, frequently accessed
- Warm storage (HDD/object store): older data
- Cold storage (Glacier, tape): archival

**Key powers of 2**: 10 KB × 10^6 objects = 10 GB; 1 MB × 10^9 = 1 PB. Memorizing these conversions prevents arithmetic errors under pressure.

Interviewers want to see: explicit per-object size assumptions, separation of data types, replication factor, and a growth rate.

Question 12

How do you estimate bandwidth in a back-of-envelope calculation?

Accepted Answer

Bandwidth estimation connects QPS to data transferred per second. It is always directional — inbound (upload) and outbound (download) are sized separately because they drive different infrastructure limits.

**Formula**: Bandwidth = QPS × Average payload size

**Step 1: Reuse QPS estimate**
From a previous estimation: 4,500 read QPS and 150 write QPS for a Twitter-like system.

**Step 2: Estimate payload size per request type**
- Tweet write: 140 chars UTF-8 ≈ 300 bytes text + metadata = ~1 KB
- Timeline read: 20 tweets per page, each ~1 KB = ~20 KB response; with media thumbnails, ~500 KB

**Step 3: Calculate**
- Inbound (write): 150 QPS × 1 KB = 150 KB/s ≈ 1.2 Mbps — negligible
- Outbound (read, text only): 4,500 QPS × 20 KB = 90 MB/s = 720 Mbps
- Outbound (read, with thumbnails): 4,500 QPS × 500 KB = 2.2 GB/s = 17.6 Gbps

**Step 4: Interpret the number**
- < 100 Mbps: single server network interface is fine
- 100 Mbps – 10 Gbps: multiple servers, 10GbE NICs, CDN for static content mandatory
- > 10 Gbps: CDN offloading is essential; origin serves only cache misses

**CDN impact**: if 90% of image requests are served from CDN edge nodes, origin bandwidth drops 10×. This is why CDN is a first-class architectural component, not an optimization.

**Key unit conversions**:
- 1 Gbps = 125 MB/s
- 1 MB/s = 8 Mbps
- 10 Gbps NIC is a common server spec — exceeding it requires multiple NICs or CDN offloading

State assumptions about average vs peak: peak bandwidth is typically 2–3× average, matching the QPS peak multiplier.

Question 13

What are the key powers of 2 every engineer should know?

Accepted Answer

Powers of 2 are the vocabulary of capacity estimation. Every storage, memory, and bandwidth number is ultimately expressed in multiples of 2^n bytes. Internalizing this table prevents unit errors under interview pressure.

**The core table:**

| Power | Value | Common name | Practical example |
|---|---|---|---|
| 2^10 | 1,024 | 1 KB | Short text field, tweet |
| 2^20 | ~1 million | 1 MB | Small image, 1 min MP3 |
| 2^30 | ~1 billion | 1 GB | Movie file, RAM on a phone |
| 2^40 | ~1 trillion | 1 TB | Laptop SSD, daily log volume |
| 2^50 | ~1 quadrillion | 1 PB | Large data warehouse |

**Why approximate with 10^3, 10^6, 10^9?**
2^10 = 1,024 ≈ 10^3. This 2.4% error compounds: 1 GB is actually 1.07 × 10^9 bytes. For estimation purposes this is irrelevant — you are trying to distinguish 'do I need 1 GB or 1 TB', not measure precisely.

**Derived conversions that appear in interviews:**
- 1 million users × 1 KB record = 1 GB
- 1 billion records × 1 KB = 1 TB
- 1 billion records × 1 MB (image) = 1 PB
- 8 bits = 1 byte → 1 Gbps = 125 MB/s
- A 64-bit integer = 8 bytes; UUID = 16 bytes; SHA-256 hash = 32 bytes

**Practical anchors:**
- L1 cache: ~32 KB; L2: ~256 KB; L3: ~8 MB
- Typical server RAM: 64–256 GB
- NVMe SSD: 1–4 TB, ~3 GB/s throughput
- S3 object max size: 5 TB

Memorise: 10, 20, 30, 40, 50 → KB, MB, GB, TB, PB. Everything else derives from this.

Question 14

What are the key latency numbers every engineer should know?

Accepted Answer

Jeff Dean's latency numbers (periodically updated as hardware improves) are the calibration tool for system design. The orders of magnitude matter more than the exact figures.

**The canonical table:**

| Operation | Latency | Notes |
|---|---|---|
| L1 cache reference | ~1 ns | |
| L2 cache reference | ~4 ns | 4× L1 |
| L3 cache reference | ~40 ns | 10× L2 |
| Main memory (DRAM) | ~100 ns | 100× L1 |
| NVMe SSD read | ~100 µs | 1,000× memory |
| HDD seek | ~10 ms | 100× SSD |
| Read 1 MB sequentially from memory | ~25 µs | |
| Read 1 MB sequentially from SSD | ~1 ms | |
| Round-trip in same datacenter | ~500 µs | |
| Cross-region network (US-EU) | ~150 ms | 300× datacenter RTT |
| DNS lookup | ~20–100 ms | Varies |

**The ratios that matter most for architecture:**

- Memory is ~1,000× faster than SSD — keep hot data in RAM
- SSD is ~100× faster than HDD — use SSD for latency-sensitive workloads
- In-datacenter network is ~300× faster than cross-region — minimize cross-region synchronous calls
- CPU operations are ~100,000× faster than disk — a tight loop hitting disk on every iteration is catastrophic

**Derived implications:**
- A cache miss that hits the database costs ~500 µs (network) + query time; Redis keeps this under 1 ms total
- Synchronous cross-region calls in the critical path add 150 ms+ — use async replication or CDN for user-facing latency
- Sequential disk reads are 10–100× faster than random — batch and sort writes (LSM trees, log-structured storage)

These numbers change as hardware evolves (NVMe is 10× faster than SATA SSD) but the relative order and ratios are stable benchmarks.

Question 15

How do you wrap up a system design answer effectively?

Accepted Answer

The wrap-up is not a summary — it is a structured hand-off that signals engineering maturity. A strong close covers four areas:

**1. Recap key design decisions with justifications**
Restate the two or three most consequential choices: 'I chose Cassandra over Postgres because our write pattern is append-only with no complex joins, and we need to scale to 1M writes/day across regions.' This demonstrates that your choices were deliberate, not default.

**2. Identify current bottlenecks and scaling limits**
Every design has a weakest link. Name it: 'As designed, the single write master becomes a bottleneck around 50K QPS. The next step would be horizontal write sharding by user_id range.' This shows forward-thinking and that you understand where the design will break before it breaks.

**3. Propose observability and operational concerns**
Mention what you would monitor: 'I would track cache hit rate, p99 latency on the feed generation service, and replication lag on secondaries. An alert on replication lag > 1s prevents stale reads from going unnoticed.' This distinguishes senior from mid-level candidates — mid-level candidates design for the happy path, senior candidates design for operations.

**4. Invite critique and alternate approaches**
Explicitly surface the tradeoffs you made: 'The main tradeoff I made was favoring write availability over strong read consistency in the social graph. If the product requires exact follower counts in real time, we would need to revisit that.' Then ask: 'What aspect would you like to explore further?' This reframes the interview as a collaborative design discussion, which is the senior engineering dynamic interviewers are testing for.

**Timing**: wrap-up should take 2–3 minutes, not 10. The interview should feel like it ended at a natural decision point, not because you ran out of time. Pace the rest of the session to leave room for it.

**What to avoid**: vague summary ('So that's the system'), re-drawing components already covered, or asking 'Did I do okay?' — keep it technical and forward-looking.

Question 16

What are the common cache invalidation strategies?

Accepted Answer

Cache invalidation is one of the hardest problems in distributed systems because you must balance consistency, availability, and performance simultaneously. The four primary strategies each fit different access patterns.

**TTL (Time-To-Live) expiration** is the simplest: every entry carries a max-age, and the cache discards it automatically. It requires no coordination with the source of truth but tolerates stale data for up to the TTL window. Use it when slight staleness is acceptable—CDN responses, DNS records, rate-limit counters.

**Event-driven (active) invalidation** purges or updates cache entries immediately when the underlying data changes. This is achieved via cache-aside updates on write, pub/sub events (e.g., Redis keyspace notifications), or explicit purge API calls. It minimizes stale reads but introduces coupling between the write path and cache management logic.

**Write-through** keeps the cache always in sync by writing to both the cache and the database synchronously on every mutation. Read latency stays low; write latency increases slightly. The cache is never stale, but every write pays the double-write cost even for data that is never re-read.

**Write-behind (write-back)** acknowledges the write to the client after updating the cache only, then flushes to the database asynchronously. This maximizes write throughput but risks data loss if the cache node dies before the flush. It is appropriate for high-throughput, loss-tolerant workloads like analytics counters.

**Cache-aside (lazy loading)** lets the application populate the cache on cache miss. On invalidation, the application simply deletes the key; the next read repopulates it. This avoids loading rarely-accessed data but can cause a thundering herd when many keys expire simultaneously—mitigated with jitter on TTLs or probabilistic early expiration.

In practice, production systems layer strategies: a short TTL provides a safety net, event-driven purges handle critical updates immediately, and write-through is reserved for hot, frequently-read data. The key engineering decision is choosing the acceptable staleness window and whether consistency must be strong, eventual, or bounded-staleness.

Question 17

How do CRUD operations map to HTTP methods?

Accepted Answer

CRUD (Create, Read, Update, Delete) maps onto HTTP methods through REST conventions. The mapping is not one-to-one—HTTP provides more verbs than CRUD has operations—and understanding the nuances prevents common API design mistakes.

**Create → POST (or PUT)**. POST to a collection URL (`POST /articles`) creates a new resource with a server-assigned ID. The response should be `201 Created` with a `Location` header pointing to the new resource. PUT to a specific URL (`PUT /articles/abc`) creates or replaces the resource at that exact URL—used when the client controls the ID (e.g., upserts).

**Read → GET**. `GET /articles` returns a collection; `GET /articles/1` returns a single resource. GET is safe and idempotent, so it can be cached and retried freely. The response is `200 OK` with the representation in the body. If the resource does not exist, return `404 Not Found`.

**Update → PUT or PATCH**. This is where most confusion lives. PUT sends a complete replacement of the resource—the client must include all fields. PATCH sends a partial update—only the changed fields. Use PUT when the client always has the full representation; use PATCH for partial edits (e.g., updating a single status field). Both return `200 OK` with the updated resource, or `204 No Content` when no body is needed.

**Delete → DELETE**. `DELETE /articles/1` removes the resource. Return `204 No Content` on success (no body needed) or `200 OK` if you return the deleted representation. A second DELETE on an already-deleted resource should return `404 Not Found`—the resource is gone.

**Status codes matter**: returning `200` for a creation instead of `201`, or returning `200` for a deletion instead of `204`, technically works but loses semantic signal. Clients that inspect status codes (reverse proxies, monitoring tools, generated SDK clients) rely on this precision.

A consistent, semantically correct CRUD-to-HTTP mapping makes APIs predictable, reduces client bugs, and enables standard tooling (caches, retry logic, API gateways) to work correctly out of the box.

Question 18

What are the best practices for API design?

Accepted Answer

Good API design is a contract between producers and consumers. A poorly designed API is expensive to change once clients depend on it. The following practices, applied consistently, produce APIs that are predictable, evolvable, and safe to build on.

**Resource-oriented URLs**: Use nouns, not verbs. `/orders` and `/orders/42` are correct; `/getOrder` is not. Nest resources only one level deep to avoid URL fragility—prefer `/orders/42/items` over `/users/1/orders/42/items`.

**Correct HTTP semantics**: Map CRUD to HTTP methods precisely (POST/GET/PUT/PATCH/DELETE). Return semantically correct status codes (201 for create, 204 for delete, 422 for validation errors, 409 for conflicts). Never return 200 with an error payload.

**Versioning**: Version from day one, before you have clients. URL versioning (`/v1/`) is the most visible and easiest to route at the gateway layer. Header versioning (`Accept: application/vnd.api.v2+json`) is cleaner but harder to test. Additive changes (new fields, new optional parameters) are non-breaking; removing or renaming fields is breaking.

**Pagination, filtering, sorting**: For collections, always paginate. Prefer cursor-based pagination over offset for large or frequently-updated datasets—offset pagination is inconsistent when items are inserted mid-page. Expose filtering via query parameters (`?status=active&sort=-created_at`).

**Authentication and authorization**: Use OAuth 2.0 / JWT for stateless auth. Never put credentials in URLs. Validate authorization at the resource level, not just the endpoint level—users should not be able to access other users' resources by ID manipulation (IDOR).

**Rate limiting**: Return `429 Too Many Requests` with `Retry-After` and `X-RateLimit-*` headers. Limit by API key or user identity, not just IP.

**Idempotency keys**: For non-idempotent POST operations that have significant side effects (payments, email sends), accept an `Idempotency-Key` header and deduplicate server-side.

**Error responses**: Return structured, consistent error bodies with a machine-readable code, a human-readable message, and optionally a documentation link. Never expose stack traces.

**Documentation**: Use OpenAPI (Swagger) to generate interactive docs and client SDKs. Treat the spec as the source of truth, not an afterthought.

Question 19

What is a write-around cache strategy?

Accepted Answer

Write-around caching is a strategy in which write operations bypass the cache entirely and go directly to the persistent storage (database, object store). The cache is only populated on cache miss during a subsequent read. This contrasts with write-through (cache and storage updated synchronously on every write) and write-back (cache updated immediately, storage updated asynchronously).

**How it works**: On a write, the application writes directly to the backing store, leaving the cache untouched. If a cached entry for the same key exists, it is either left stale (to expire via TTL) or explicitly invalidated. On a subsequent read, a cache miss triggers a read from storage and a cache population.

**When to use it**: Write-around is optimal when written data is unlikely to be re-read in the near term. Classic use cases include:
- **Log ingestion and analytics**: Log records are written once and read rarely or in bulk aggregate queries, not individually.
- **Bulk imports / batch ETL**: Large volumes of data loaded into a database that won't be queried immediately don't benefit from cache population—they would evict hot entries.
- **Media upload pipelines**: A newly uploaded video or image is written to object storage but rarely fetched in the seconds after upload.

**Advantages**: Prevents cache pollution—the cache is not filled with cold data that will never be re-read, preserving cache space for frequently-accessed data. Reduces write latency compared to write-through (only one write, to storage, not two).

**Disadvantages**: First reads after a write incur a cache miss penalty. If the same data is frequently read shortly after write (e.g., a user reading back a record they just created), write-around causes unnecessary latency. In those cases, write-through or application-level cache population on write is preferable.

**Comparison with write-through**: Write-through keeps the cache always warm for recently written data at the cost of double-write overhead. Write-around is cheaper on write but cold on immediate re-read. The choice depends on the read-after-write access pattern.

A hybrid approach: use write-around for bulk/background writes, but explicitly populate the cache for user-facing writes where read-after-write is the dominant pattern.

Question 20

What is the relationship between reliability, fault tolerance, and redundancy?

Accepted Answer

Reliability, fault tolerance, and redundancy are related but distinct concepts that together form the foundation of resilient system design. Conflating them leads to over-engineering in some areas and blind spots in others.

**Reliability** is the probability that a system performs its intended function correctly over a specified period under specified conditions. It is typically expressed as availability (e.g., 99.9% uptime = 8.7 hours downtime/year). Reliability is the outcome you're optimizing for.

**Fault tolerance** is the system's ability to continue operating correctly in the presence of one or more component failures. A fault-tolerant system degrades gracefully rather than failing catastrophically. Fault tolerance is a design property—it is achieved through specific techniques.

**Redundancy** is the duplication of critical components so that if one fails, another takes over. Redundancy is the primary mechanism for achieving fault tolerance. Types include:
- **Active-active redundancy**: All redundant components handle traffic simultaneously. Capacity scales with node count; failover is instant. Requires all nodes to stay in sync.
- **Active-passive (standby)**: One primary handles traffic; standby waits. Simpler state management, but failover takes time and the standby is wasted capacity until needed.
- **N+M redundancy**: N components required to operate, M extra for failures. RAID 5 (N+1 parity), RAID 6 (N+2).

**The relationship**: Redundancy enables fault tolerance; fault tolerance is what makes the system reliable under adverse conditions. A system can have redundancy without being fault-tolerant (if failover is not automated) or be fault-tolerant in ways that don't require redundancy (graceful degradation—serving cached data when a dependency is down).

**Beyond redundancy—fault tolerance techniques**:
- **Circuit breakers**: Prevent cascading failures by stopping calls to a failing dependency and returning a fallback response.
- **Bulkheads**: Isolate failures to a subsystem—a slow database query pool does not exhaust thread pools for other services.
- **Graceful degradation**: Serve reduced functionality rather than total failure (show cached product data when the inventory service is down).
- **Retry with backoff and jitter**: Tolerate transient faults without overwhelming a recovering service.

Reliability targets (SLOs) drive the engineering trade-off: moving from 99.9% to 99.99% requires eliminating single points of failure and automating failover, which significantly increases infrastructure cost and operational complexity.

System Design Fundamentals Interview Questions — What Senior Engineers Need to Know

All 20 Questions

How to Prepare

Related Topics

Test Your Knowledge