Consumer Group and Rebalance

A conventional Kafka consumer group shares subscribed partitions among its members. Each partition is assigned to at most one member at a time; one member can own several partitions. Separate groups track progress independently and can read the same retained records.

A rebalance changes assignments as membership or subscription conditions change. A new owner resumes from the applicable committed progress. If the old owner completed output but failed before committing, work can repeat. Assignment exclusivity therefore does not mean exactly one processing attempt.

Coordinate in-flight work, revoked assignments, and commits according to the client protocol. Prevent an obsolete worker from applying uncontrolled external effects. More members than assigned partitions leaves idle members. This description concerns conventional consumer groups, not share groups that distribute records with individual acknowledgements.

Reference: Kafka fundamentals lab.