Correlation

Correlation summarizes how strongly two variables move together in one number between \(-1\) and \(1\). The Pearson coefficient is the standard version, and the word doing the most work in its definition is linear: as the forecasting literature puts it, “correlation measures the extent of a linear relationship between two variables.”

That single qualifier accounts for most of what the number misses, and everything in this entry follows from it or from the causal question at the end.

Four things the coefficient hides

• Non-linear relationships. A variable that rises then falls with another can have a correlation near zero while being perfectly determined by it. Zero correlation means no linear association, not no association.
• Outliers. The coefficient is built from products of deviations, so a few extreme points can create a strong correlation in unrelated data or destroy a real one. This is why the number should never be read without the scatter plot.
• Restricted range. Correlation computed on a narrow slice of one variable is usually much weaker than on the full range — a familiar trap when a sample is already filtered, for example to paying customers.
• The level of aggregation. Correlation between daily totals, between users, and between events are three different quantities, and they can differ in sign. Which one answers your question is a decision to make before computing anything.

Two related notes. For simple OLS with an intercept, nonconstant variables, and evaluation on the training sample, squaring the coefficient gives the centered training coefficient of determination — see R² — which is a smaller number than intuition suggests: \(r = 0.5\) accounts for 25% of the variance. And correlation between successive values of one series over time is the same idea applied to lags, which is temporal autocorrelation and has its own consequences for inference.

Why it does not support an intervention claim

The slogan “correlation is not causation” is true and too weak to be useful, because it does not say what else could be producing the association. The causal-inference literature states the position more precisely: “the best explanation for an association between treatment and outcome in an observational study is not necessarily a causal effect of the treatment on the outcome.“

Three alternatives are always available, and naming which one you are ruling out is the useful move.

• A common cause. Something drives both. Engaged customers use more features and spend more; the feature may do nothing.
• Reverse direction. The outcome drives the supposed cause. Heavy spenders adopt the feature because they are heavy spenders.
• Selection. How units entered the data produces the association. Comparing only users who stayed a month conditions on something the treatment may have affected.

Getting from association to an intervention claim requires something correlation does not contain. “Causal inference from observational data requires two elements: data and identifiability conditions,” and the conditions are assumptions that the data cannot confirm — in an ideal randomized experiment the identification conditions are supplied by the design. Actual experiments still need uncertainty estimates and checks for interference, attrition, measurement differences, and nonadherence. See causal effect for the conditions, A/B testing for the design that supplies them, and causal inference for the field.

Using it well

Correlation is a good screening tool and a poor conclusion. It is worth computing to find candidates worth examining, to check whether two metrics are redundant, and as a sanity check on a pipeline — if two measures of the same thing suddenly stop being correlated, investigate definitions, population and range changes, noise, and the measurement pipeline before assigning a cause.

Three habits make it safe. Plot before you report, because the coefficient cannot distinguish the shapes that produce it. Say what the unit of observation is, since the aggregation level changes the answer. And treat it as one number with an uncertainty of its own — a correlation from thirty points is a weak statement, which is easiest to convey with an interval; see confidence intervals and, where a closed form is awkward, bootstrap. Where the goal is to model a relationship rather than summarize it, that is regression.

References

• Hyndman & Athanasopoulos, Forecasting: Principles and Practice (3rd ed.), §2.8 “Autocorrelation”
• Hernán & Robins, Causal Inference: What If (2020)
• Google, Site Reliability Engineering, Effective Troubleshooting.