Event Time and Processing Time
Event time is the timestamp representing when the modeled event happened. Processing time is when a particular operator handles it. Ingestion time records arrival at a specified entry point. Name those points: arrival at a broker and processing after a queue are different observations.
A payment happens at 09:04, reaches a broker at 09:07, and is processed at 09:10. An event-time total for [09:00, 09:05) associates it with that window. A processing-time count places it in the operator’s later interval. Both can answer valid questions, but only the first measures payments that happened in the earlier interval. Replaying tomorrow does not change the intended payment time.
Event timestamps can be wrong because of device clocks, units, timezones, or a mistaken field. A timestamp on a row is not proof of its meaning. Preserve the source value and arrival evidence, normalize using an explicit timezone, and review implausible future values before they advance event-time progress. Processing-time behavior can change during replay even when the records are identical.
Reference: Flink time concepts.
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
