Causal Trees

A causal tree is a regression tree whose leaves hold estimated treatment effects rather than predicted outcomes, and whose splits are chosen to find variation in those effects. The target is the conditional average treatment effect, \(\tau(x) = E[Y(1) - Y(0) \mid X = x]\), the difference between what would happen to a unit with features \(x\) under treatment and under control. For a randomized experiment with constant assignment probability, a leaf can be estimated by the difference of group means, \(\hat{\tau}(x) = \hat{\mu}(1, x) - \hat{\mu}(0, x)\). Every unit in that leaf receives the same estimated average effect; this is not an observed individual effect. In observational data, simply placing units in the same leaf need not remove confounding.

How the splits differ from a regression tree

A conventional tree splits to reduce the error of predicting \(Y\), and it can do that because every row carries a \(Y\) to compare a prediction against. A causal tree would like to split to reduce the error of predicting \(\tau\), and cannot, for the reason Athey and Imbens state plainly: “The causal effect is not observed for any individual unit, and so we do not directly have a ground truth”. Their term for the mean squared error of treatment effects is that it is infeasible — the quantity exists, and no dataset contains the labels needed to compute it.

Their way around this is to estimate the infeasible criterion rather than abandon it. Leaf-level effects can be estimated on two independent samples, and the product of the two estimates has an expectation that recovers the part of the mean squared error that depends on the partition. That yields a splitting criterion built entirely from observable quantities, and it is the reason a causal tree is not simply a regression tree run on some transformed column.

The distinction matters more than the machinery. A tree fitted to outcomes finds features that predict who responds; a causal tree finds features that predict whom the treatment changes. In targeting work the two sets of features can be nearly disjoint, and using the first where the second is needed produces campaigns aimed at people who would have converted regardless. That failure has its own name in the marketing literature and is covered under uplift models.

Honesty

Athey and Imbens define the property their method is built around: an estimator is honest “if it does not use the same information for selecting the model structure (in our case, the partition of the covariate space) as for estimation given a model structure”. In practice the training data are divided, one part used to grow and cross-validate the tree and another used only to estimate the effect in each leaf.

The reason is a bias that conventional trees carry silently. An algorithm that searches for the split with the largest apparent difference will tend to place extreme sample values together, so the leaf means it reports are more extreme than the same leaves would show in fresh data. For prediction that overstatement is a familiar overfitting cost. For treatment effects it is worse, because the headline output of the tree is a set of subgroup effects that a reader will treat as findings, and the confidence intervals around them will be centered on numbers the search chose for being large.

Their simulations quantify both sides. Honest estimation gives what they call nominal coverage for 90 percent confidence intervals, and every honest cell in their results table falls between 0.89 and 0.91. For the adaptive versions they report coverage ranging between 74 and 84 percent, and the worst cell in the table is 0.69 — an interval that claims 90 percent and delivers 69. The cost is that the tree is built and the effects estimated on smaller samples, which raises the mean squared error of the treatment effects by 7 to 22 percent for their preferred method. These figures describe the paper’s simulation settings, not guaranteed coverage or a fixed cost for every dataset. The practical tradeoff is between using more data to search for a partition and reserving independent data for its effect estimates.

The criterion, and why it penalizes small leaves

Anticipating honest estimation changes what a good split looks like, and the criterion is adjusted accordingly. It has two terms: the average of squared leaf effects, which rewards a partition for finding real heterogeneity, and a subtracted term that accumulates, over the leaves, the within-leaf variance of treated and control outcomes divided by their respective shares of the sample, scaled by \(1/N^{tr} + 1/N^{est}\). The authors summarize it as criteria that “reward a partition for finding strong heterogeneity in treatment effects and penalize a partition that creates variance in leaf estimates”.

Two consequences are worth carrying away. A split that separates units with genuinely different effects is rewarded even if it barely changes predicted outcomes. And a split can be worth making for variance alone: if a covariate drives the outcome without changing the effect, splitting on it makes the leaves more homogeneous and the treated and control means within them less noisy, so the effect estimates improve although the effects themselves are identical. Variance reduction is therefore another reason for a split, which is why a causal tree’s structure should not be read as a map of where the effect varies.

The transformed-outcome shortcut

There is a simpler route that is often met first. In an experiment with a known and constant treatment probability \(p\), define \(Y^*_i = Y_i (W_i - p) / (p(1 - p))\), where \(W_i\) is the treatment indicator. Then \(E[Y^* \mid X = x] = \tau(x)\), so an off-the-shelf regression tree fitted to \(Y^*\) estimates treatment effects, and no new software is needed. Athey and Imbens keep it as a benchmark and note its cost: the method uses the treatment indicator only to build the transformed column, and the leaf average of \(Y^*\) equals the difference-of-means estimate only when the treated fraction inside that leaf is exactly \(p\). Chance variation in leaf composition therefore feeds straight into the estimates, and the method is less efficient than one that keeps the treated and control groups distinct.

What it does not buy you

A causal tree relaxes an assumption about functional form. It does not relax the assumptions that make a causal quantity estimable from the data at all. Athey and Imbens maintain unconfoundedness throughout — treatment assignment is independent of the potential outcomes given the covariates — along with no interference between units, consistency, and overlap. Randomized assignment with a nonzero chance of either arm delivers design-level overlap and unconfoundedness. A small realized leaf can still lack enough observations from one arm. It does not deliver no interference: if one user’s treatment changes another user’s outcome — a referral, a shared inventory, a social feed — randomization leaves that intact, and the design and estimand must address those spillovers. Cluster randomization can help when interference is contained within the chosen clusters; it does not automatically remove interference across them. In observational data unconfoundedness becomes a claim about the world as well, and tree fitting alone cannot establish it. Counts and assignment probabilities help diagnose support, but cannot prove that no confounder is missing.

Question	Who answers it	What happens if it is wrong
Is the difference of group means a causal effect here?	The design, or an identification argument — see causal inference	Every leaf reports a confounded comparison, stated to three decimal places
Is there a treated and an untreated unit in this region of feature space?	Assignment probabilities and adequate data in both arms	A within-leaf difference is undefined; software may reject the split or use an explicitly different estimator
What shape does the effect take across features?	The tree	Poor fit, visible in validation
How uncertain is a leaf’s effect?	Honest estimation, and the interval built from the estimation sample	Intervals that claim 90 percent and cover as little as 69

The middle row is the one that produces confident nonsense. A leaf containing treated units and no controls, or the reverse, has no comparison to make, and its simple difference-of-means estimate is undefined. Any fallback needs its own explanation.

For example, suppose past high-value customers were more likely to receive a promotion. A broad leaf can contain both arms yet still mix different types of customers in different proportions. Its raw response-rate difference combines selection with treatment. The observational extension in Athey and Imbens adjusts leaf estimates, for example by propensity-score weighting, under conditional unconfoundedness and overlap. Honesty separates model selection from estimation; it does not substitute for that adjustment or for an identification argument.

A worked reading

The following leaves are illustrative. A retailer randomizes a promotional email and fits a causal tree on age, income and browsing history, with conversion as the outcome.

Leaf	Estimated effect	Treated / control units	What it supports
Young, lower income	-2 percentage points	3,100 / 3,050	A candidate suppression rule, if it survives validation
Middle-aged, higher income	+15 percentage points	2,900 / 2,880	A candidate target if uncertainty, costs, and validation support it
Older customers	+0.2 percentage points	4,000 / 3,960	Small; an interval is needed before calling it zero

The counts belong in the table for the same reason the effects do. A leaf with 40 treated and 35 control units can report +15 points just as easily, and the difference between the two situations is not visible in the effect column. The interval around each estimate, computed on the estimation sample, is what separates a subgroup finding from a subgroup guess; confidence intervals covers the general reading.

From a tree to a forest

A single tree is unstable in the usual way — a small change in the data moves a split and rearranges the subgroups below it — and instability is worse when the signal is a difference between groups. Wager and Athey extend the idea to an ensemble, a causal forest, and their contribution is not only smoothing: under their assumptions on identification, sampling, and forest construction they show their estimator is pointwise consistent for the true treatment effect and has “an asymptotically Gaussian and centered sampling distribution”, with a practical method for building asymptotic confidence intervals around its estimates. A forest is one option for estimating conditional effects more smoothly, while the single tree retains its own role: a partition a person can read, state as a rule, and argue about.

Tree-based uplift modeling has a separate industrial lineage. Radcliffe and Surry’s production trees split on a significance-based criterion instead, and they report only limited success when they tried using a rank-based quality measure as the split criterion. Different criterion, same problem: a split has to be judged by something that responds to the difference between the groups. The estimator families are surveyed under causal ML.

References

• Susan Athey and Guido Imbens, “Recursive partitioning for heterogeneous causal effects”, Proceedings of the National Academy of Sciences 113(27), 2016 (open-access copy at PubMed Central)
• Stefan Wager and Susan Athey, Estimation and Inference of Heterogeneous Treatment Effects using Random Forests, arXiv:1510.04342; Journal of the American Statistical Association 113(523), 2018
• Nicholas J. Radcliffe and Patrick D. Surry, Real-World Uplift Modelling with Significance-Based Uplift Trees, Portrait Technical Report TR-2011-1, Stochastic Solutions White Paper, 2011