Kafka — auto.offset.reset

KafkaMidkafka

The Question

What does auto.offset.reset do in Kafka?

What a Strong Answer Covers

• earliest vs latest defined
• when it applies.

Senior-Level Answer

`auto.offset.reset` is a consumer configuration that determines where Kafka begins reading messages when a consumer group has no committed offset for a partition — or when the committed offset is out of range (e.g., it was deleted due to log retention). The two primary values are `earliest` and `latest`.

With `earliest`, the consumer starts from the oldest available message on the partition — the beginning of the retained log. This is appropriate for new consumers that need to process all historical data, such as a freshly deployed service reading an event log to rebuild state. With `latest`, the consumer starts from the current end of the log, receiving only messages produced after the consumer first started. This is typically used for real-time feeds where historical data is irrelevant.

The third option, `none`, causes the consumer to throw an exception if no committed offset is found, forcing you to handle the bootstrap case explicitly in code. It is rarely used in production but can be valuable in strict operational environments where starting from an unexpected offset would be a bug.

Critically, `auto.offset.reset` only applies when there is no valid committed offset for the group. If a committed offset exists — even a stale one — Kafka uses it and ignores this setting entirely. This trips up engineers who reset a consumer group or deploy a new group expecting a specific behavior but forget that existing committed offsets take precedence.

A common production scenario where this matters: your retention policy deletes log segments, and a lagging consumer's last committed offset no longer exists in the log. The consumer's offset is now out of range. With `earliest`, it resets to the beginning of available data (potential reprocessing of recent messages); with `latest`, it skips ahead to current (potential data loss). Knowing which is appropriate for your use case is an architectural decision, not just a configuration detail.

What Separates a 2/3 from a 3/3

Common Mistakes

• Thinking auto.offset.reset applies on every consumer startup — it only applies when there is no committed offset for that group.
• Not knowing about the 'none' option and what it does.
• Forgetting that log retention can invalidate previously committed offsets, triggering the reset behavior unexpectedly.
• Assuming 'latest' means the consumer skips only new messages after restart — it means the first time the group runs with no offsets.

Follow-Up Questions

• Your consumer group has been down for 3 days and your retention is 2 days. What happens when it restarts with auto.offset.reset=earliest? — Committed offset is out of range (log segment deleted). Consumer resets to the earliest available offset — 2 days of data, not the original starting point.
• You deploy a new consumer group alongside an existing one to replay all events. Which setting do you use? — earliest — the new group has no committed offsets so it starts from the beginning of the retained log.
• What does auto.offset.reset=none do and when would you use it? — Throws OffsetOutOfRangeException when no committed offset exists — forces explicit handling, useful when silent resets would be a correctness bug.
• How would you force a consumer group to re-read from the beginning without changing auto.offset.reset? — Use kafka-consumer-groups.sh --reset-offsets --to-earliest to reset committed offsets manually.

Related Questions

• Kafka — acks=0/1/all
• Kafka — Why Is It Fast
• Kafka — Consumer Lag
• Kafka — ISR
• Kafka — Consumer Group