Kafka — ISR

KafkaSeniorkafka

The Question

What is the ISR (In-Sync Replicas) in Kafka?

What a Strong Answer Covers

• ISR definition
• removal when lagging
• interaction with min.insync.replicas.

Senior-Level Answer

The In-Sync Replicas (ISR) list is a dynamic set maintained per partition by the Kafka broker acting as partition leader. A replica is considered "in-sync" if it has fetched all messages from the leader within the window defined by `replica.lag.time.max.ms` (default 30s). If a follower falls behind — due to network issues, GC pauses, or slow disk — it is removed from the ISR.

The ISR is critical for durability guarantees. When a producer sends with `acks=all` (or `acks=-1`), the leader waits until every replica in the current ISR has acknowledged the message before returning success to the producer. This means ISR shrinkage directly impacts write latency — a slow replica that stays in ISR causes the producer to wait for it.

During leader election, Kafka by default only allows replicas in the ISR to be elected as the new leader. This prevents data loss: a non-ISR replica may be missing messages the previous leader had already acknowledged. The `unclean.leader.election.enable` setting (default false) controls whether out-of-ISR replicas can be elected — enabling it trades consistency for availability.

Operationally, you can inspect ISR status via `kafka-topics.sh --describe` or through broker metrics (`UnderReplicatedPartitions`, `IsrShrinksPerSec`). A consistently shrinking ISR is a leading indicator of broker health issues and should trigger investigation before it affects availability.

The interplay between ISR, `min.insync.replicas`, and `acks` defines your actual durability contract. ISR is the mechanism; the other two settings are the policy knobs that tell Kafka how to enforce that contract under degraded conditions.

What Separates a 2/3 from a 3/3

Common Mistakes

• Confusing ISR with the full replica set — ISR is a dynamic subset, not all assigned replicas.
• Thinking ISR only matters for reads — it exclusively governs write acknowledgment and leader election.
• Not knowing that a lagging ISR member slows producer writes when `acks=all` is set.
• Assuming `unclean.leader.election.enable=true` is safe — it can cause data loss by electing a replica missing committed messages.

Follow-Up Questions

• What happens to write throughput if one replica in a 3-node ISR becomes very slow? — With acks=all, the leader waits for the slowest ISR member — explore the trade-off between removing it from ISR vs. degraded throughput.
• How does `replica.lag.time.max.ms` interact with ISR membership? — Candidate should know this is the timeout after which a follower is evicted from ISR for not fetching.
• What is the risk of setting `unclean.leader.election.enable=true`? — An out-of-ISR replica elected as leader may be missing messages already acknowledged to producers — data loss.
• How would you monitor ISR health in production? — UnderReplicatedPartitions JMX metric, IsrShrinksPerSec, and kafka-topics.sh --describe are key tools.

Related Questions

• Kafka — acks=0/1/all
• Kafka — Why Is It Fast
• Kafka — Consumer Lag
• Kafka — Consumer Group
• Kafka — Rebalance