Holdback Group
A randomized holdback can estimate a policy’s continuing effect when assignment, follow-up, treatment history, and the outcome window support the comparison.
-
-
Prescriptive Analytics
A system that recommends without a declared objective has one anyway — chosen by whoever wrote the code, and invisible to everyone else.
-
Descriptive and Diagnostic Analytics
Breaking a decline down by region and channel produces candidates, not causes. The gap between the two is the hardest step in analytics.
-
Types of Analytics and the Questions They Answer
“Why did revenue drop” and “what will revenue be next quarter” look like neighbouring questions. They need different methods and different evidence.
-
Treatment Effect
The outcome with the treatment minus the outcome without it, for a unit or on average. Which average, on which scale, for whom: the estimand an estimate needs.
-
Causal Inference
Causal inference from observational data needs data and identifiability conditions. Choosing a method does not establish the identification assumptions.
-
Incremental Recovery Rate (IRR)
How much more recovery an intervention produced than doing nothing — a rate, in percentage points, and not a quantity until a population is attached.
-
Causal ML (Causal Machine Learning)
Causal ML relaxes the shape of the model, not the assumptions that make a comparison causal. Those stay exactly as strong as before.
-
Incremental Gain
The extra outcome produced inside one slice of a ranking. Computed from response rates rather than counts, and not additive across slices when the treated share differs between them.
-
Cumulative Incremental Gain (CIG)
The incremental outcome collected down to a depth of a ranking. Accumulating slice by slice gives a different answer from computing it directly.
-
Qini Curve
Radcliffe’s incremental gains chart: treated responders minus what an equal treated group would have produced untreated. Its baselines, and why they matter.
-
Causal Trees
A regression tree whose leaves estimate treatment effects. The criterion that makes the splits, the sample split called honesty, and what it does not fix.
-
Uplift Curve
Cumulative incremental outcome against the share targeted. Why its vertical axis scales a difference of response rates into an incremental response count, and where to cut the campaign.
-
Correlation
Pearson correlation measures linear association only. A coefficient near zero can sit on top of a perfect curved relationship.
-
Causal Effect
Causal inference from observational data needs two things: data and identifiability conditions. Choosing a method does not establish the identification assumptions.
-
Regression Models
A regression predicts an outcome; it does not tell you what would happen if you changed an input. Those are different questions with different requirements.
-
AUUC (Area Under the Uplift Curve)
The area under the uplift curve, reduced to a single number — by at least three different conventions that do not agree, on the same model and the same data.
-
Qini Coefficient
A single number for an uplift model’s ranking, from the area above the random diagonal. The normalization changes it more than most model differences do.
-
Uplift
The difference between the outcome with the treatment and the outcome without it. Why a response model does not estimate it, and what the number is attached to.
-
Causal Inference for Machine Learning
Prediction asks what Y will be; intervention asks what Y would be if you set X.
-
Intention-to-Treat Analysis
In a randomized offer, only some people take it up, and those who do are not typical. Intention-to-treat compares everyone as assigned, which keeps the randomization and estimates the effect of the offer.
-
Difference-in-Differences
Compare the change in a treated group with the change in a comparison group. The method removes a permanent gap and a shared trend, and relies on the groups having moved in parallel.
-
Measuring Platform Adoption and Developer Friction
Adoption counts say who touched a platform, not whether work got easier. This article builds workflow measures, qualitative research, and baselines, and states what they cannot prove.