Event Time and Processing Time

Event time is the timestamp representing when the modeled event happened. Processing time is when a particular operator handles it. Ingestion time records arrival at a specified entry point. Name those points: arrival at a broker and processing after a queue are different observations.

A payment happens at 09:04, reaches a broker at 09:07, and is processed at 09:10. An event-time total for [09:00, 09:05) associates it with that window. A processing-time count places it in the operator’s later interval. Both can answer valid questions, but only the first measures payments that happened in the earlier interval. Replaying tomorrow does not change the intended payment time.

Event timestamps can be wrong because of device clocks, units, timezones, or a mistaken field. A timestamp on a row is not proof of its meaning. Preserve the source value and arrival evidence, normalize using an explicit timezone, and review implausible future values before they advance event-time progress. Processing-time behavior can change during replay even when the records are identical.

Reference: Flink time concepts.


Discover more from Insightful Data Lab

Subscribe to get the latest posts sent to your email.

Similar Posts

Questions, corrections, or additional insights?

This site uses Akismet to reduce spam. Learn how your comment data is processed.