Poisson Process

A Poisson process is the standard model for events occurring in time when each occurrence is independent of the others. Requests arriving at a server, calls reaching a switchboard, particles registering on a detector. It is described by a rate — expected events per unit time — which may itself vary with time.

Its defining behavior is what happens when you count. The number of events in an interval is Poisson distributed with mean equal to the integral of its rate over that interval (rate times interval length when the rate is constant), and counts in non-overlapping intervals are independent. Pooling n repeated observation windows multiplies the expected count accordingly: for a window of width Δ at local rate θ, the pooled count has mean nΔθ.

The property that makes it analytically useful is that its variance equals its mean. Both are the same number, so the spread of the counts is not a free parameter to be estimated separately — it is determined by the level. Several estimation methods depend on this: it converts an unknown amount of noise into something readable off the counts themselves.

That same identity is the assumption most often violated in practice. Real event streams cluster: a failure triggers retries, one user loops, a batch job fires a thousand records at once. Clustered arrivals are overdispersed — the variance of the counts exceeds the mean — and any method resting on variance-equals-mean is then computing from a premise the data contradict.

Compare counts across equal-width intervals with comparable expected rates, or examine residuals after modeling the changing rate. A mixture of independent Poisson counts with different means can have variance above its mean even when increments remain independent. Excess variation beyond an adequate rate model motivates checking clustering, dependence, and model misspecification; simply pooling more windows does not establish the model.

See it in use in Choosing Histogram Bins for Time-Indexed Events.

References

• Shimazaki & Shinomoto, bin size selection method