Changelog Topic
Keep state locally and back it with a log: simple to reason about, and the reason a restart takes as long as the log is long.
-
-
Stream Processing Semantics: Time, State, and What Exactly-Once Actually Covers
In streaming, a number is never simply right. It is right as of a boundary you chose, over data that had arrived by then, and it may change tomorrow.
-
Allowed Lateness
An engine that silently drops late records is not misbehaving. Keeping a window open to accept them costs state, which is why the default is to keep nothing.
-
Log Compaction
Time-based retention forgets by age; compaction forgets by key. One keeps a recent window of events, the other keeps the latest value for everything.