Revenue net of treatment cost

Revenue net of treatment cost is what a campaign earned after paying for itself. It exists as a named quantity because the gross figure flatters: a campaign that produced \$20,000 of incremental revenue and cost \$18,000 to run is not a \$20,000 success, and a report that leads with the first number is describing activity rather than value.

What is netted against what

Before the arithmetic, fix the accounting boundary and apply each item once. Three overlaps cause most of the double counting: a discount already reflected in the observed revenue and then subtracted again as a cost; negative responders already netted out by the uplift estimate and then added as destroyed value; and a per-contact cost charged on everyone contacted when the cost is only incurred by those who redeem.

Revenue net of treatment cost is incremental revenue minus treatment cost. For a profitability decision, revenue also has to cover the variable costs of the goods behind it. Revenue minus treatment cost and contribution minus treatment cost are different amounts, and the second is the decision figure used here, so the chain runs through the margin: incremental revenue, times the contribution margin, minus the treatment cost.

Skipping the second row is the usual error, and it is not small: at a 30 percent margin it overstates the result by more than three times the true contribution. The first row matters too. Take the difference as treated minus control, per customer, then scale by \(N\). Differencing the arms’ totals instead estimates the incremental revenue scaled to one arm’s size — not the treated arm’s own revenue, which includes the baseline — and with unequal arms it estimates neither, because the smaller arm books less revenue regardless of the campaign. Fix what \(N\) stands for before reading the result. The components are set out under incremental revenue and treatment cost.

Break-even

Reading the net figure as a threshold is often more useful than reading it as a total. Each contacted customer has to generate enough incremental revenue that the margin on it covers the contact. At a 30 percent contribution margin and \$2 per contact, that is \$6.67 of incremental revenue per customer contacted.

Written that way the question becomes answerable before the campaign runs, and comparable across campaigns with different costs. In the worked example under incremental revenue, a campaign that clearly lifts conversions produces an estimated \$0.32 of incremental revenue per customer — twenty times short of that threshold, so a blanket send loses money even though the treatment demonstrably works. The response is not a better estimate but a narrower target: concentrating the spend on the part of the ranking where the effect is, which is what total incremental benefit works through, and what Radcliffe and Surry do by attaching cost and value to an evaluation curve and reading the profit-maximizing volume off it.

Where the uncertainty sits

The two sides of the subtraction are not known with the same confidence, and the asymmetry is extreme. The cost is mostly arithmetic: contacts multiplied by a rate that is on an invoice. The benefit is an estimate of a difference between two group means, and for revenue that estimate is imprecise even on large samples — in the same worked example the point estimate is \$19,463 with a 95 percent interval from -\$32,652 to +\$71,578, both in dollars of incremental revenue.

So the interval on the net figure is, to a close approximation, the interval on the incremental contribution shifted down by the cost: scale the revenue interval by the contribution margin first, then subtract the cost. Subtracting the cost from the revenue interval directly leaves it on the revenue scale. Two consequences follow. A net figure reported without an interval hides all of its uncertainty, since the precise-looking cost contributes almost none of it. And a net figure whose interval spans zero has not established that the campaign paid for itself, whatever the point estimate says; Kohavi and colleagues note that the interval doubles as the test in exactly this way.

Radcliffe and Surry attach the same caveat to their own profit calculations: the figures assume “that the uplift estimates are accurate”, and “in general, uplift cannot be estimated as accurately as a purchase rate can be”.

Comparing two campaigns

The natural use of the net figure is to choose between options, and the natural mistake is to treat similar uplift estimates as equal uplift. Overlapping intervals do not establish that the two are equal — testing the difference requires an interval on the difference, which on shared data needs the covariance and is usually narrower than the overlap suggests. Where the difference genuinely cannot be resolved, that does not make the expected benefits equal. Choosing the cheaper campaign then assumes equal expected benefit as a tie-break, which is a decision under uncertainty rather than a finding that they perform alike, and it should be stated as one. Otherwise compare each option’s expected net value, using its own estimated benefit and its own cost. If the difference matters enough to justify the traffic, the comparison worth running is a direct one between the two treatments rather than two separate measurements against no treatment.

A second comparison is easy to skip and often decisive: the campaign against not running it. A net figure of \$5,000 is only an argument for the campaign if the alternative use of the same budget and the same audience attention returns less, and the audience attention is a real cost that never appears in the cost line.

References

• Nicholas J. Radcliffe and Patrick D. Surry, Real-World Uplift Modelling with Significance-Based Uplift Trees, Portrait Technical Report TR-2011-1, Stochastic Solutions White Paper, 2011
• Ron Kohavi, Roger Longbotham, Dan Sommerfield and Randal M. Henne, “Controlled experiments on the web: survey and practical guide”, Data Mining and Knowledge Discovery 18(1), 2009