Incremental Recovery Rate (IRR)

The incremental recovery rate is how much more often an outcome is recovered when an intervention is applied than when it is not. It is the uplift of a recovery outcome: with \(R\) for recovery, \(T\) for the treated condition and \(C\) for the control,

\[ \text{IRR} = P(R \mid T) - P(R \mid C) \]

The term is used where the desirable outcome is a return to good standing rather than a purchase: debt collection, where recovery means repayment; healthcare, where it means clinical improvement; and retention, where it means a lapsed customer coming back. The structure is the same as any uplift measurement — a difference between two groups that differ only by chance — and the entry under uplift covers the general form.

The acronym collides

IRR is also the standard abbreviation for internal rate of return, and the two live in the same documents. A collections program is evaluated on how much extra repayment it produces and on whether the investment in it clears a hurdle rate, and a slide carrying both is not unusual. Write the term out at first use and the ambiguity disappears; leave it as three letters in a finance-adjacent report and a reader will supply the wrong expansion.

A rate, not a quantity

A bank tests a new collections script. These figures are illustrative: the treated group repays at 35 percent and the control group at 28 percent.

Rows three and four are the same fact and differ by a factor of the baseline; reporting one as “7 percent” and the other as “25 percent” without saying which scale is meant invites exactly the confusion that conversion rate uplift deals with. Row five is a different kind of statement: it multiplies a rate measured on the test population by a population that was never tested, which is a projection rather than a measurement. It is reasonable when the accounts are drawn from the same pool and unreasonable when the test ran on recent delinquencies and the roll-out reaches aged ones.

What the window decides

Recovery is an event in time, and the measurement window is part of the definition rather than a reporting detail. A script that produces faster repayment without producing more of it shows a large effect at 30 days and none at 180. The reverse also happens: an intervention with a slow build looks inert in a short window. Neither pattern is visible in a single number, so the window belongs next to it.

Competing outcomes complicate the same measurement. An account that is written off, settled at a discount, or enters a payment plan has left the pool without recovering in the sense being counted, For a defined binary outcome of recovery by a fixed date, a competing event that prevents that recovery can legitimately count as nonrecovery. That estimates the recovery risk in the presence of competing events, including any treatment effect on those events. Censoring is different: recovery status may not be observed for the full window, so treating a censored case as a known nonrecovery needs justification. State the outcome and window, distinguish competing events from censoring, and choose an analysis for that estimand.

How precise it is

An incremental recovery rate is a difference between two observed proportions and carries the standard error of one. Kohavi and colleagues give the interval in its standard form, and note that it doubles as the test: when zero falls inside it, the intervention has not been shown to work. Collections and clinical tests often run on a few hundred accounts or patients, which is precisely where a seven-point difference can fail to be distinguishable from zero — confidence intervals, two-proportion z-test and sample size cover the arithmetic and the design.

The comparison is causal here because assignment was random. Where a collections team applied the new script to the accounts they judged most promising, the raw difference measures that judgment rather than the script, and recovering the effect requires adjusting for what drove the choice — see random targeting strategy. And Radcliffe and Surry’s caution about uplift generally applies here too: the difference of two rates is estimated less accurately than either rate, so a program’s effect is usually known far less precisely than its recovery rate.

Using it to target

The reason the metric belongs in uplift work rather than in ordinary reporting is that recovery effort is expensive and finite. Ranking accounts by how likely they are to repay puts effort where repayment would have happened anyway; ranking by how much the intervention changes repayment puts it where the effort matters. That is the distinction drawn under uplift models, and the per-account estimate it produces is an uplift score whose outcome happens to be recovery.

References

• Nicholas J. Radcliffe and Patrick D. Surry, Real-World Uplift Modelling with Significance-Based Uplift Trees, Portrait Technical Report TR-2011-1, Stochastic Solutions White Paper, 2011
• Ron Kohavi, Roger Longbotham, Dan Sommerfield and Randal M. Henne, “Controlled experiments on the web: survey and practical guide”, Data Mining and Knowledge Discovery 18(1), 2009