Question 1

What is consistent hashing and why is it used?

Accepted Answer

Consistent hashing is a distributed hashing technique designed to minimize the number of keys that must be remapped when the number of nodes changes. It is foundational to distributed caches (Memcached, Redis Cluster), distributed databases (Cassandra, DynamoDB), and load balancers.

**The problem with naive hashing:**

With N servers, naive modular assignment is `server = hash(key) % N`. Adding or removing a server changes N, which changes the computed slot for almost every key — in expectation, nearly all K keys move. For a distributed cache this means a near-total cache miss storm; for a database it means massive data migration.

**The consistent hashing solution:**

Map both nodes and keys onto a **hash ring** — a logical circle from 0 to 2^32 (or 2^64). Hash each node to a position on the ring. To look up the server for a key, hash the key to a ring position and walk clockwise until you find the first node — that node owns the key.

When a node is added: only the keys between the new node and its predecessor need to move — approximately K/N keys on average. When a node is removed: only the keys it owned move to its successor — again approximately K/N. All other keys are unaffected.

**Virtual nodes (vnodes):**

A single physical node hashed to one ring position creates uneven distribution — different arc lengths mean different load. The standard fix is **virtual nodes**: each physical node is hashed to multiple positions on the ring (e.g., 150 vnodes per physical node). This statistical sampling evens out the load distribution. It also makes rebalancing more granular — adding a physical node gradually takes over vnodes from many others rather than one large arc.

**Consistent hashing in practice:**

- **Cassandra:** Uses vnodes. The ring is divided into token ranges. Each node is responsible for its token range. A new node joins by splitting existing ranges.
- **Redis Cluster:** Uses 16,384 hash slots (not a ring, but a similar concept). Slots are redistributed on topology changes.
- **Memcached client libraries:** Implement consistent hashing client-side to route requests to the correct server.
- **Load balancers:** Route requests from the same source IP to the same backend server (session affinity) by hashing source IP on the ring.

**Limitations:** Consistent hashing assumes uniform key distribution in the hash space. Hot keys (a small set of keys receiving disproportionate traffic) still cause hot spots on specific nodes — vnodes mitigate but don't eliminate this. Additional strategies: key splitting, client-side sharding bypass, or read replicas for hot keys.

The time complexity for lookup is O(log N) using a sorted data structure (binary search on the sorted list of node positions) — not O(N) linear scan.

Question 2

How do you implement rate limiting and what algorithms exist?

Accepted Answer

Rate limiting is a traffic control mechanism that restricts how many requests a client can make within a time window. It protects services from abuse, prevents resource exhaustion, and ensures fair usage across tenants.

The four main algorithms each make different trade-offs. **Fixed window** divides time into discrete buckets and counts requests per bucket. It's simple to implement but suffers from the boundary burst problem: a client can fire 2× the limit by straddling two windows (e.g., 100 requests at 00:59 and 100 more at 01:01).

**Sliding window log** keeps a timestamp log of every request and counts entries within the rolling window. It's precise but memory-intensive—O(requests) space per client. **Sliding window counter** approximates the sliding window by combining the current and previous fixed-window counts weighted by overlap. It's O(1) space with ~0.1% error, making it the practical default.

**Token bucket** gives each client a bucket that refills at a constant rate (e.g., 10 tokens/second, capacity 50). Each request consumes one token; excess requests are dropped or queued. It allows controlled bursting up to bucket capacity—good for APIs where brief spikes are legitimate.

**Leaky bucket** processes requests at a constant outflow rate regardless of arrival rate, smoothing traffic absolutely. It cannot burst, which makes it ideal for protecting downstream systems that can't handle spikes.

In distributed systems, rate limiting state must be shared. A single Redis instance with atomic INCR/EXPIRE works for small scale. For higher scale, use Redis Cluster or a dedicated rate-limit service. The key design decisions: where to store counters (Redis, Memcached, or in-process with gossip-based sync), what the key is (IP, user ID, API key, tenant), and how to handle Redis unavailability (fail open vs. fail closed).

Sliding window counter in Redis is implemented with two keys per window and an atomic Lua script, ensuring no race conditions. Response headers like `X-RateLimit-Limit`, `X-RateLimit-Remaining`, and `Retry-After` are standard UX. HTTP 429 Too Many Requests is the correct status code.

Question 3

What is a CDN and how does it work?

Accepted Answer

A Content Delivery Network (CDN) is a geographically distributed network of servers (edge nodes or PoPs—Points of Presence) that cache and serve content closer to end users than a single origin server can.

The request flow works as follows: a user's browser performs a DNS lookup for `assets.example.com`. The CDN's authoritative DNS responds with the IP of the nearest edge node, determined by anycast routing or GeoDNS. The browser connects to that edge node. If the edge has the asset cached (cache hit), it serves it directly—often in single-digit milliseconds. On a cache miss, the edge fetches from the origin (or a mid-tier cache), stores the response according to cache-control headers, and returns it to the user.

CDNs reduce latency by eliminating round trips to a distant origin. They also reduce origin load dramatically—a popular asset served with a high cache-hit ratio means the origin receives a tiny fraction of actual user requests. This provides implicit DDoS mitigation, since the CDN absorbs volumetric traffic.

Cache behavior is controlled by HTTP headers: `Cache-Control: max-age`, `s-maxage` (CDN-specific TTL), `Surrogate-Control`, and `Vary`. `Vary: Accept-Encoding` tells the CDN to cache separate copies for gzip vs. brotli. `Vary: Cookie` essentially disables caching for authenticated content—a common pitfall.

Cache invalidation is the hard problem. Options include: TTL expiry (lazy, no active invalidation), URL versioning/fingerprinting (`app.a3f7c.js`—permanent cache, invalidation by deploy), and API-based purge (Cloudflare, Fastly, Akamai all expose purge APIs for instant invalidation by URL or tag).

Modern CDNs also support edge compute (Cloudflare Workers, Fastly Compute@Edge), allowing request manipulation, A/B testing, and auth at the edge without hitting the origin. They handle TLS termination at the edge, reducing handshake latency for users far from the origin.

Key metrics: cache-hit ratio (higher is better), TTFB (Time to First Byte), and origin offload percentage. A well-configured CDN typically achieves 90%+ cache-hit ratio for static assets.

Question 4

What is the difference between L4 and L7 load balancers?

Accepted Answer

L4 and L7 load balancers operate at different layers of the OSI model and serve different purposes.

L4 load balancing operates at the transport layer. It makes routing decisions based on TCP or UDP header data — source/destination IP, ports, and protocol type. It does not inspect the payload. When a connection arrives, the load balancer selects a backend and forwards all packets for that connection. Examples: AWS Network Load Balancer (NLB), HAProxy in TCP mode, Linux IPVS.

L7 load balancing operates at the application layer. It terminates the client connection, parses the full request (HTTP headers, URL path, cookies, request body), and makes intelligent routing decisions. This enables path-based routing (/api/* to API servers, /static/* to CDN), session affinity via cookies, A/B testing, rate limiting, and WAF integration. Examples: AWS Application Load Balancer (ALB), NGINX, Envoy.

SSL/TLS termination is a key differentiator. L7 balancers typically terminate TLS, decrypting traffic to inspect HTTP content. L4 balancers can pass through TLS without terminating it.

Performance: L4 balancers are faster and handle more connections per second — minimal processing, microsecond latency. L7 balancers add low-millisecond latency for parsing and buffering, but can optimize with HTTP/2 multiplexing and connection pooling.

Connection handling: L4 maintains a single client-to-backend connection (transparent). L7 maintains two separate connections: client-to-LB and LB-to-backend, requiring X-Forwarded-For to preserve client identity.

Use L4 for raw throughput and non-HTTP protocols (gaming, database connections). Use L7 for content-aware routing and API gateway functionality.

In modern architectures, both are often used together — L4 at the edge distributing to multiple L7 balancers, which then perform intelligent routing.

Question 5

What are 2PC and Saga patterns for distributed transactions?

Accepted Answer

Distributed transactions arise when a single business operation must update data across multiple services or database nodes atomically. Two-Phase Commit (2PC) and the Saga pattern are the two dominant approaches, with fundamentally different consistency guarantees and failure models.

**Two-Phase Commit (2PC)** uses a coordinator to achieve distributed atomicity. Phase 1 (Prepare): the coordinator sends a PREPARE to all participants, which each execute the transaction locally, write to a write-ahead log, and vote YES or NO. Phase 2 (Commit/Abort): if all vote YES, the coordinator sends COMMIT; if any vote NO, it sends ABORT. All participants apply or roll back accordingly.

The critical flaw: 2PC is a blocking protocol. If the coordinator crashes after sending PREPARE but before sending COMMIT, participants are in an uncertain state—they've locked resources and cannot proceed without the coordinator. This is the 'in-doubt' window, and it can block indefinitely. It also requires all participants to be available simultaneously, making it impractical across microservices with independent availability SLAs. 2PC is appropriate for coordinating nodes within a single database system (e.g., XA transactions across two PostgreSQL instances in the same data center) but is rarely used across services.

**Saga pattern** decomposes a long-running transaction into a sequence of local transactions, each publishing an event or message that triggers the next step. Consistency is achieved through compensating transactions—each step has an inverse that undoes its effect if a later step fails.

There are two coordination styles. **Choreography**: each service listens for events and decides what to do next—decentralized, lower coupling, but harder to trace the overall workflow. **Orchestration**: a central saga orchestrator (or state machine) explicitly calls each service in sequence and handles failures—easier to observe and debug, but introduces a coordination point.

Sagas provide eventual consistency, not atomicity. Compensations are not rollbacks—they issue new transactions that semantically undo prior work (e.g., issuing a refund vs. reversing a payment). This means the system can be in an inconsistent intermediate state during execution, which is acceptable for many business workflows but requires careful design of idempotency and compensation logic.

Use 2PC when you need strong ACID atomicity across a small number of co-located resources and can tolerate the blocking risk. Use Saga when you need to coordinate across microservices with independent deployments, high availability requirements, or long-running workflows where blocking is unacceptable.

Question 6

Is exactly-once delivery possible and how do you achieve it?

Accepted Answer

Exactly-once delivery means a message is processed by the consumer exactly one time — neither lost nor duplicated. In theory it is achievable; in practice it requires specific infrastructure support and careful application design.

First, clarify the two failure modes. **At-most-once**: the producer sends without waiting for an acknowledgment, or the consumer commits its offset before processing. Messages can be lost but never duplicated. Suitable for metrics or logging. **At-least-once**: the producer retries on timeout and the consumer commits after processing. Messages are never lost but can be processed multiple times on crashes. Most systems default here.

Exactly-once delivery requires solving both sides simultaneously. The producer must not introduce duplicates when it retries, and the consumer must not reprocess a message it already handled.

**Producer side — idempotent producers**: Kafka's idempotent producer assigns each message a sequence number. The broker deduplicates retries within a session using (producer ID, partition, sequence). This eliminates producer-side duplicates but only within a single session — a broker restart resets the producer ID.

**Consumer side — idempotent processing**: The application must make the processing operation idempotent. Common techniques: (1) database upsert keyed on a message ID — inserting the same order twice is a no-op; (2) storing the processed offset alongside the state update in a single atomic transaction; (3) a deduplication table that records seen message IDs with a TTL.

**Kafka transactions** combine both: the producer writes to multiple partitions and commits the consumer offset atomically within a single transaction. Consumers set `isolation.level=read_committed` to skip uncommitted messages. This gives exactly-once semantics within the Kafka ecosystem.

**Distributed transactions (2PC)** can coordinate writes across heterogeneous systems (Kafka + Postgres) but introduce coordinator availability as a bottleneck and are slow. Most teams prefer the outbox pattern instead: write the event to an outbox table in the same DB transaction as the business write, then have a separate process relay it to Kafka. This reduces cross-system coordination to at-least-once + idempotent consumer.

The honest answer: exactly-once is achievable within a single system (Kafka, or a single DB with transactions), but across heterogeneous distributed systems it requires careful design trade-offs, and most production systems settle for at-least-once with idempotent consumers.

Question 7

How does leader election work in distributed systems?

Accepted Answer

Leader election is the process by which a group of distributed nodes agrees that exactly one node — the leader — is responsible for coordinating writes, holding a lock, or running a singleton process. Without a reliable election mechanism, you get split-brain: two nodes both believe they are leader, causing conflicting writes and data corruption.

The core challenge is that any election algorithm must handle the impossibility result (FLP): in an asynchronous network with even one faulty process, you cannot guarantee both safety (one leader at a time) and liveness (an election always completes). Practical systems relax the asynchrony assumption using timeouts.

**Raft** is the most widely understood algorithm. Nodes start as followers. If a follower receives no heartbeat from a leader within an election timeout (150–300ms typically), it transitions to candidate, increments its term, and sends `RequestVote` RPCs to all peers. A node votes for at most one candidate per term. The candidate that collects a majority of votes becomes leader for that term, immediately sending heartbeats to suppress new elections. Raft guarantees that only a node with the most up-to-date log can win — voters reject candidates whose log is behind theirs.

**ZooKeeper (ZAB protocol)** uses a similar approach: ephemeral sequential znodes. Each node creates a znode under `/election/`; the node with the lowest sequence number is the leader. All others watch the znode immediately preceding theirs. When the leader's ephemeral node disappears (session timeout), the next-lowest znode holder becomes leader. This avoids the thundering herd of all followers watching a single node.

**etcd** exposes a lease-based leader election API. A candidate writes a key with a TTL-backed lease. If it wins the compare-and-swap (key did not previously exist), it is leader. The leader must renew the lease periodically; if it crashes, the lease expires and another candidate wins.

**Fencing tokens** are critical for safety. Even after a leader loses its election, it may still believe it is leader due to a network partition (the "stale leader" problem). Every operation should include a monotonically increasing fencing token issued by the election system; the storage layer rejects requests with a lower token than it has seen. This prevents a stale leader from corrupting state.

In practice: use etcd or ZooKeeper for production leader election rather than implementing your own. Rolling your own timing-based election is a common source of split-brain bugs.

Question 8

What is feed fanout and what are the push vs pull approaches?

Accepted Answer

Feed fanout is the mechanism by which a post from one user is delivered to the feeds of all their followers. At small scale this is trivial; at Twitter/Instagram scale, a single post from a celebrity with 50 million followers must fan out to 50 million feed entries — a massive write amplification problem.

**Fanout-on-write (push model)**: When a user publishes a post, a background job immediately writes the post ID into each follower's feed cache (usually a Redis sorted set keyed by timestamp). Feed reads are O(1) — just fetch the pre-built list. The cost is write amplification: one post triggers millions of cache writes. For most users with <10K followers this is fine. For celebrities, a single tweet could overwhelm your write path.

**Fanout-on-read (pull model)**: The feed is computed lazily when the user opens their app. The system fetches the list of accounts the user follows, retrieves recent posts from each, merges and sorts them. No write amplification, but read latency is high and proportional to following count. Not viable for users who follow thousands of accounts.

**Hybrid approach** (what Twitter and Instagram actually use): Apply push to normal users (following count < threshold, e.g. 1M followers). Apply pull to celebrity accounts. When building a feed, merge the pre-built push cache with a real-time pull from celebrity accounts. The threshold is tunable — when a celebrity posts, you skip the fanout and let readers pull at read time.

Storage: each user's feed is typically a Redis sorted set of post IDs with the score as a timestamp. Only IDs are stored, not full post content — a separate lookup fetches post data. Feed length is capped (e.g., 800 entries) and older entries are evicted.

**Backfill and consistency**: new followers don't automatically get historical posts unless you backfill. Unfollows must purge entries — or tolerate stale entries with a grace window. These edge cases are often handled asynchronously.

At the infrastructure level, fanout jobs run as async workers (Celery, Kafka consumers, Sidekiq). Slow workers cause delivery lag — a post may appear in some feeds seconds before others, which is acceptable eventual consistency for social feeds.

Question 9

What are RPO and RTO and why do they matter?

Accepted Answer

RPO and RTO are the two fundamental metrics that define a system's disaster recovery requirements. They are not technical metrics — they are business requirements that translate into technical architecture decisions.

**Recovery Point Objective (RPO)** is the maximum acceptable amount of data loss measured in time. If your RPO is 1 hour, your recovery strategy must ensure you can restore the system to a state no older than 1 hour ago. RPO is determined by asking: "If we lost all data after a certain point, how much data loss can the business tolerate?" A payment processor might have RPO = 0 (zero data loss). A blog might accept RPO = 24 hours.

**Recovery Time Objective (RTO)** is the maximum acceptable downtime — the time from a failure event to full service restoration. If your RTO is 4 hours, you must be able to detect the failure, execute recovery, and restore normal service within 4 hours. RTO is determined by asking: "How long can the business operate without this system?" An e-commerce site during Black Friday might have RTO = 5 minutes. An internal analytics dashboard might have RTO = 24 hours.

These two metrics drive different technical investments:

- **Backup frequency**: To achieve RPO = 1h, you need backups every hour at minimum. Continuous WAL archiving (point-in-time recovery) approaches RPO = seconds.
- **Replication**: Synchronous replication achieves RPO = 0 but adds write latency. Asynchronous replication improves write performance at the cost of RPO > 0.
- **Hot standby vs. cold standby**: A hot standby (fully replicated, running) achieves RTO in seconds/minutes. A cold standby (restore from backup) might take hours — fine for RTO = 4h, not for RTO = 15min.
- **Multi-AZ vs. multi-region**: Multi-AZ addresses data-center failures with low RTO/RPO. Multi-region addresses regional outages at the cost of higher replication lag (higher RPO) and more complex failover.

A common mistake is conflating RPO and RTO or designing infrastructure without explicit targets. Organizations should establish RPO/RTO per service tier, test recovery procedures regularly, and document the gap between target and actual achieved metrics. Having a 4-hour RTO target but a recovery procedure that actually takes 8 hours is a liability.

Question 10

What is the circuit breaker pattern and what are its three states?

Accepted Answer

The circuit breaker pattern is a fault-tolerance mechanism that prevents a service from repeatedly calling a dependency that is failing or slow, which would otherwise waste resources, increase latency, and propagate failures upstream. The name comes from electrical circuit breakers: when current exceeds a safe threshold, the breaker trips to protect the circuit.

A circuit breaker wraps calls to an external dependency (database, third-party API, microservice) and tracks the outcome of recent calls. Based on failure rate or count, it transitions through three states:

**Closed (normal operation)**: All calls pass through to the dependency. The breaker counts failures in a rolling window (e.g., last 10 calls or last 60 seconds). If the failure rate exceeds a threshold (e.g., 50%), the breaker **trips** to Open.

**Open (failing fast)**: All calls are immediately rejected without contacting the dependency — they fail fast with a fallback response (cached data, default value, error to client). A timer starts. This prevents thread-pool exhaustion and gives the dependency time to recover. After the timeout (e.g., 30 seconds), the breaker transitions to Half-Open.

**Half-Open (probing for recovery)**: A limited number of probe requests are allowed through to the dependency. If the probes succeed (failure rate drops below threshold), the breaker resets to Closed. If probes fail, it immediately trips back to Open and restarts the timer. This prevents premature recovery that would immediately overload a struggling service.

Two failure metrics are commonly used: **count-based** (trip after N consecutive failures) and **rate-based** (trip when failure % exceeds threshold within a time window). Rate-based is generally more accurate under variable traffic — high failure counts in absolute terms may represent a low failure rate during peak traffic.

Fallback strategies: return a cached response, return a default/empty response, queue the request for later processing, or return a user-visible error with appropriate messaging ("Service temporarily unavailable").

In practice: libraries like Hystrix (deprecated), Resilience4j (JVM), Polly (.NET), and PyBreaker implement this pattern. Service meshes like Istio implement circuit breaking at the infrastructure layer, removing the need for application-level instrumentation. Key configuration parameters: failure threshold, time window, open-state timeout, half-open probe count.

Question 11

What are the common load balancing algorithms?

Accepted Answer

Load balancing algorithms determine how incoming requests are distributed across a pool of backend servers. The right choice depends on request duration variance, session affinity requirements, and backend heterogeneity.

**Round Robin** distributes requests sequentially across all backends. It is simple, stateless, and works well when requests have similar processing times and servers have equivalent capacity. Weighted Round Robin extends this by assigning a weight to each backend, directing proportionally more traffic to higher-capacity servers.

**Least Connections** routes each new request to the server with the fewest active connections. This adapts well to variable request duration — if some requests take much longer than others, round robin would overload the servers handling those slow requests. Least Outstanding Requests is a variant that tracks in-flight requests rather than established connections, useful for HTTP/2 multiplexed streams.

**IP Hash (Sticky)** hashes the client IP address to consistently route a given client to the same backend. This provides session affinity without application-level session sharing, but it can cause uneven distribution when a small number of IPs generate disproportionate traffic, and it breaks if a backend is added or removed (use consistent hashing to mitigate this).

**Consistent Hashing** maps both requests (by a key such as URL, user ID, or IP) and servers onto a hash ring. When a server is added or removed, only requests that were mapped to that server are remapped — minimizing cache invalidation in systems like distributed caches or CDN origin selection.

**Random** selects a backend uniformly at random. With Power of Two Choices (P2C), two backends are sampled randomly and the one with fewer active connections is chosen — this achieves near-optimal balance with O(1) overhead, used in Envoy and Nginx.

**Resource-Based / Adaptive** algorithms use real-time metrics (CPU, memory, response latency) reported by agents on each backend. This is the most accurate but most complex, requiring a health reporting infrastructure.

In practice, least connections or P2C is the default choice for most microservice deployments. Round robin is sufficient for stateless, uniform request loads. Consistent hashing is essential when backend affinity must survive partial pool changes.

Question 12

What is the difference between a push CDN and a pull CDN?

Accepted Answer

CDNs (Content Delivery Networks) reduce latency by caching content at edge nodes geographically close to users. Push and pull CDNs differ in *who* initiates the content placement at the edge.

In a **pull CDN**, the edge node fetches content from the origin server on the first request — a **cache miss**. The CDN then stores the response and serves subsequent requests from cache until the TTL expires (driven by `Cache-Control: max-age`). The operational model is minimal: you set TTLs in your origin responses and let the CDN handle placement. The downside is the **cold start problem**: the first request to each edge node after a deploy or TTL expiry hits the origin, adding latency. Under high traffic, many simultaneous misses can cause a **thundering herd** stampede to the origin. Pull CDNs are the default for most web assets because they are self-managing — Cloudflare, Fastly, and CloudFront all operate as pull CDNs by default.

In a **push CDN**, you explicitly upload content to the CDN's storage (via API or rsync) before any user requests it. The CDN distributes your uploaded content to all edge nodes. All requests are served from edge storage — there is no origin fetch. This is appropriate for large static assets that change infrequently and must be served with zero-miss latency globally: software downloads, large media files, game patches. AWS CloudFront with S3 as origin approximates a push CDN; pure push CDNs include MaxCDN's push zones and Rackspace CDN.

The trade-offs: push CDNs require you to manage the content lifecycle — you must purge or re-upload on change, and you pay for storage at all edge nodes whether or not the content is ever requested in that region. Pull CDNs are lazily populated but require a working origin and may serve stale content until TTL expires or explicit purge is triggered.

For a typical SaaS application: use a pull CDN for dynamic API responses (short TTL or `Cache-Control: private`) and HTML, and use a pull CDN with long TTL + content-hashed filenames for static assets (JS, CSS, images). Reserve push CDN semantics for large media delivery where zero-miss SLAs are required and content lifecycle is fully controlled by your deployment pipeline.

Question 13

How does CDN cache invalidation work?

Accepted Answer

A CDN caches origin responses at geographically distributed edge nodes so that subsequent requests are served locally without hitting the origin. Cache invalidation is the mechanism by which stale copies at those edge nodes are replaced or discarded.

**TTL-based expiration** is the default mechanism. The origin sets `Cache-Control: max-age=<seconds>` (or the legacy `Expires` header). Each edge node honours this value independently; once a cached object's age exceeds max-age, the next request triggers a conditional revalidation (`If-None-Match` / `If-Modified-Since`) or a full re-fetch. Because edge nodes are globally distributed and may have cached the object at different times, expiration is not simultaneous—different users may see stale content for different durations within the TTL window.

**API-based purge** (also called instant purge or cache bust) lets operators send an explicit invalidation request—typically a DELETE or POST to the CDN's management API—that propagates a purge signal to all edge nodes. Most major CDNs (Cloudflare, Fastly, Akamai) guarantee propagation within seconds. Purge can target individual URLs, URL patterns (surrogate keys / cache tags), or entire zones. Cache tags are particularly powerful: a single tag like `product:42` can be attached to all pages that embed product 42, and one tag purge clears all of them atomically.

**URL versioning (cache-busting)** sidesteps invalidation entirely by embedding a content hash or build ID in the asset URL (e.g., `app.a3f9c1.js`). The old URL is simply abandoned; it expires naturally. This is the preferred approach for static assets in CI/CD pipelines because it eliminates the race condition between deploy and purge propagation.

**Stale-while-revalidate** is a middle-ground directive: the CDN serves the stale object immediately while fetching a fresh copy in the background. This trades brief staleness for zero-latency updates and is ideal for content that is frequently updated but where millisecond accuracy is not required (news feeds, dashboards).

The hard problem is the window between a deploy and full purge propagation—requests hitting un-purged edges can return stale HTML referencing already-deleted asset URLs. The canonical mitigation is to deploy assets before HTML (atomic immutable asset deploy, then HTML purge), ensuring that any HTML version—old or new—references valid assets.

Question 14

How does load balancer failover work?

Accepted Answer

Load balancer failover is the process by which a load balancer detects that a backend instance has become unhealthy and automatically removes it from the pool of eligible targets, redirecting traffic to healthy instances without manual intervention.

**Health checks** are the detection mechanism. Active health checks: the load balancer proactively sends periodic probe requests (HTTP GET to `/health`, TCP connect, or ICMP ping) to each backend. If a backend fails to respond within a timeout or returns a non-2xx status for N consecutive checks, it is marked unhealthy and removed from rotation. Passive health checks: the load balancer monitors real request responses—if a backend returns 5xx errors or times out above a threshold, it is marked unhealthy. Passive checks have lower overhead but detect failures only when real traffic is affected.

**Failover sequence**: (1) Health check detects failure. (2) Load balancer marks instance unhealthy, stops routing new requests to it. (3) In-flight requests to the failed instance may be retried (if the method is idempotent and the load balancer is configured for retries) or returned as errors to the client. (4) Traffic redistributes across remaining healthy instances. (5) If all instances fail, the load balancer returns 503 Service Unavailable.

**Session persistence (sticky sessions)** complicates failover: if sessions are pinned to a specific backend via cookie or IP hash, a backend failure breaks all sessions on that node. Mitigation: store session state externally (Redis, database) so any backend can serve any session.

**Connection draining (deregistration delay)**: before removing an instance (for scale-in or deployment), the load balancer stops routing new requests but allows in-flight requests to complete, up to a configurable timeout. This prevents 5xx spikes during rolling deploys.

**Active-passive vs. active-active**: In active-passive, the secondary load balancer is idle until the primary fails (detected via heartbeat/VRRP). In active-active, multiple load balancers share traffic via DNS round-robin or anycast. Active-active has no cold-start failover penalty but requires all nodes to handle full load.

The practical tuning levers are health check interval, consecutive failure threshold, and connection drain timeout—too sensitive causes flapping; too lenient delays failover during real outages.

Distributed Systems Interview Questions — What Senior Engineers Need to Know

All 14 Questions

How to Prepare

Related Topics

Test Your Knowledge