What is uplift@k?
- uplift@k is a performance metric used in uplift modeling / causal ML.
- It measures the incremental effect (uplift) achieved if we only target the top k% of customers (or samples) ranked by the model’s predicted uplift score.
- In other words: “If I only contact the top k% of customers, how much additional impact do I get compared to not contacting them?”
Formula
For the top k% of the ranked list:
$\text{Uplift@k} = \frac{y^T_{(k)}}{n^T_{(k)}} – \frac{y^C_{(k)}}{n^C_{(k)}}$
Where:
- $y^T_{(k)}$: Total outcomes (e.g., purchases) of the treatment group within the top k%
- $n^T_{(k)}$: Number of treatment samples in the top k%
- $y^C_{(k)}$: Total outcomes of the control group within the top k%
- $n^C_{(k)}$: Number of control samples in the top k%
So uplift@k = difference in average outcome between treatment and control in the top-k% segment.
Intuition
- A random targeting strategy would give a small or zero uplift (since treatment and control would perform similarly).
- A good uplift model ensures that in the top k% segment, treatment leads to much better outcomes than control.
- That difference is captured by uplift@k.
Example
Suppose you have 10,000 customers, and you target the top 20% (k=20%) = 2,000 customers according to your model.
- Treatment group purchase rate = 15%
- Control group purchase rate = 10%
$\text{Uplift@20\%} = 0.15 – 0.10 = 0.05$
Interpretation: In the top 20% selected by the model, treatment increases the purchase probability by 5 percentage points compared to control.
Use Cases
- Marketing: If you send promotions only to the top k% customers, uplift@k tells you how much incremental sales you would generate.
- Healthcare: If you administer treatment only to the top k% of patients (those most likely to benefit), uplift@k measures the incremental recovery rate.
- Recommendation systems: If you recommend items only to the top k% users predicted to respond, uplift@k measures the extra engagement/CTR gained.
Comparison: Uplift vs Uplift@k
| Metric | Uplift | Uplift@k |
|---|---|---|
| Definition | Average treatment effect across the entire population | Incremental effect within the top k% segment |
| Purpose | Measures overall treatment effect | Evaluates the model’s ranking quality (how well it identifies responsive users) |
| Use Case | Checking whether the treatment works in general | Deciding which subset of users to target for best ROI |
Key Takeaway
- uplift = “How effective is treatment overall?”
- uplift@k = “How effective is my model at selecting the best subset of users to treat?”
