Uplift Random Forests

An uplift random forest is an ensemble of decision trees grown to estimate the effect of a treatment rather than the outcome itself. The target is the same conditional effect an uplift score reports, \(\Delta(x) = P(Y = 1 \mid T = 1, x) - P(Y = 1 \mid T = 0, x)\), The trees use effect-oriented split criteria and estimate treated–control response differences in leaves; the forest aggregates those effect estimates rather than ordinary outcome predictions.

Zhang, Li and Liu describe the construction in four steps: bootstrap samples are drawn from the training data with replacement, each is projected onto a fixed number of randomly selected covariates, an uplift tree is grown on each, and a new unit’s uplift is the average of the trees’ predictions. Only the third step differs from Breiman’s original algorithm, and that is where all the design decisions live.

What the trees split on

An ordinary tree splits to reduce the impurity of the outcome. An uplift tree splits to find regions where the treatment does something different, and the literature offers several ways to say what that means.

The best-established family, from Rzepakowski and Jaroszewicz, scores a split by how much it increases the divergence between the treated and control outcome distributions. causalml states the criterion as a gain, \(D_{\text{gain}} = D_{\text{after split}}(P^T, P^C) - D_{\text{before split}}(P^T, P^C)\), with three choices of divergence between the treated response rate \(p\) and the control rate \(q\):

• Kullback–Leibler: \(p \log(p/q) + (1 - p)\log\!\big((1 - p)/(1 - q)\big)\)
• Euclidean: \(2(p - q)^2\)
• Chi-squared: \((p - q)^2/q + (p - q)^2/(1 - q)\)

The Euclidean version has become the usual default, and Zhang, Li and Liu report the reason: Rzepakowski and Jaroszewicz “argued that the Euclidean criterion is superior because it is more stable than the other criteria and has the important property of being symmetric”. Symmetry matters because the Kullback–Leibler divergence is not symmetric in its two arguments, so which group is named first changes the score; and stability matters because the chi-squared form divides by \(q\), which explodes when the control response rate in a candidate node is small.

Within the children, not between them

A description that circulates widely is that uplift trees split to maximize the difference in treatment effect between the two branches. That is one approach and not the main one. Zhang, Li and Liu draw the distinction explicitly: earlier tree-based methods “aim to find the split that maximises the difference between the estimated CATEs of the left and right child nodes, whereas UpliftDT aims to maximise the estimated CATE within each child node”. The Euclidean criterion they give is

\[ C_{Eu} = \frac{n_L}{n}\,\hat{\tau}_L^2 + \frac{n_R}{n}\,\hat{\tau}_R^2 \]

a size-weighted average of the squared effect inside each child. This is a raw child score; selecting a split by gain also compares it with the parent score and may apply normalization. For example, with the same treated share in both equally weighted children and effects of 0.2 in each, the child score is 0.04 and the parent effect is also 0.2, giving a parent score of 0.04 and zero raw gain. Large equal effects alone do not establish a useful split. Separating positive and negative effects or isolating a high-effect region can raise the criterion; arm composition, estimation variation, and the implementation affect that comparison. The criterion also generalizes to a \(p\)-way split by summing the same terms, which is how these trees handle categorical features without binarizing them.

The difference-based criteria still exist as options. causalml implements delta-delta-p, \(\big|\big(P^T(y \mid a_0) - P^C(y \mid a_0)\big) - \big(P^T(y \mid a_1) - P^C(y \mid a_1)\big)\big|\), along with a normalized variant, an interaction-tree criterion based on a squared \(t\) statistic, a causal-inference-tree criterion based on a likelihood ratio, and contextual treatment selection. The documentation notes that these are restricted to binary trees and, for several of them, to two-class problems — a real constraint when the treatment has more than two arms.

Why averaging helps more here

Ensembling a classifier is a familiar variance reduction. Ensembling an uplift tree buys more, and the reason is structural: the quantity being modeled is a difference between two probabilities, which is harder to fit than either probability, so individual trees disagree more and the ensemble is more diverse. Sołtys, Jaroszewicz and Rzepakowski studied bagging and random forests for uplift and found the gains substantially larger than in ordinary classification, with bagged uplift trees significantly better than single uplift trees and competitive with uplift random forests.

The cost is stated equally plainly by Zhang, Li and Liu: ensemble methods “lose the interpretability possessed by the tree-based methods and have higher time complexity”. A single uplift tree gives a partition a person can read out as targeting rules and argue about; a forest gives a per-customer number and no rules at all. Where the output is a ranking to act on, that trade is usually worth making. Where the output is a segment definition for a campaign team, it is not.

Two lineages

Wager and Athey’s contribution on the right-hand column is not smoothing but inference: under unconfoundedness, the causal forest is “pointwise consistent for the true treatment effect” and has “an asymptotically Gaussian and centered sampling distribution”, with a practical method for building intervals around its estimates. If the question is which customers to treat at a fixed volume, either lineage answers it. If the question is how large the effect is in a named subgroup, and how sure you are, the honest construction described under causal trees is what supports the second half of that.

What it needs, and what to check

• Both arms, everywhere you act. A leaf containing treated units and no controls has no comparison to make, and the forest returns a number rather than a refusal. Counting both arms per reported segment is the cheapest guard available.
• Randomized assignment, or an identification argument. The forest relaxes assumptions about functional form and none about identification — see causal inference.
• Evaluation on held-out data. No row carries the quantity being predicted, so accuracy is not available; what can be checked is whether the ranking separates realized outcomes, using the uplift curve and the Qini coefficient.
• A comparison against the simple alternatives. Treating everyone, treating nobody, and a two-model baseline are all cheaper, and a forest that does not beat them on realized value is an expensive way to reach the same decision — see uplift models.

The method has a documented track record on the kind of problem it was built for. Guelman, Guillén and Pérez-Marín applied it to insurance renewals, with 8,249 treated and 3,719 control customers whose policies were due, That is the scale of their application, not a universal minimum: required data depend on outcome variability, effect size, allocation, splitting, and desired precision.

References

• Weijia Zhang, Jiuyong Li and Lin Liu, A unified survey of treatment effect heterogeneity modelling and uplift modelling, ACM Computing Surveys, arXiv:2007.12769, 2021
• causalml documentation, Methodology — tree-based uplift splitting criteria
• Stefan Wager and Susan Athey, Estimation and Inference of Heterogeneous Treatment Effects using Random Forests, arXiv:1510.04342; Journal of the American Statistical Association 113(523), 2018
• Piotr Rzepakowski and Szymon Jaroszewicz, “Decision trees for uplift modeling with single and multiple treatments”, Knowledge and Information Systems 32(2), 2012
• Michał Sołtys, Szymon Jaroszewicz and Piotr Rzepakowski, “Ensemble methods for uplift modeling”, Data Mining and Knowledge Discovery 29(6), 2015
• Leo Guelman, Montserrat Guillén and Ana M. Pérez-Marín, “Uplift random forests”, Cybernetics and Systems 46(3–4), 2015