1. Definition
- Conversion Rate Uplift measures the relative improvement (or decrease) in conversion rate between a treatment (variant) and a control (baseline).
- It answers:
“By what percentage did the new variant increase (or decrease) conversions compared to the baseline?”
2. Formula
Let:
- $CR_A$ = conversion rate of control
- $CR_B$ = conversion rate of treatment
(a) Absolute Uplift
$\text{Absolute Uplift} = CR_B – CR_A$
Measured in percentage points.
(b) Relative Uplift
$\text{Relative Uplift} = \frac{CR_B – CR_A}{CR_A} \times 100\%$
Measured as a percentage improvement over baseline.
3. Examples
Example 1 – Positive Uplift
- Control conversion rate: $CR_A = 5\%$
- Treatment conversion rate: $CR_B = 6\%$
- Absolute uplift = $6\% – 5\% = 1\%$ point
- Relative uplift = $\frac{6 – 5}{5} \times 100 = 20\%$
Interpretation: The treatment increases conversions by +20% relative to control.
Example 2 – Negative Uplift
- Control: $%CR_A = 10\%$
- Treatment: $CR_B = 9.5\%$
- Absolute uplift = –0.5% points
- Relative uplift = $\frac{9.5 – 10}{10} \times 100 = –5\%10$
Interpretation: The treatment reduced conversions by 5%.
4. How It’s Used in A/B Testing
- Primary success metric: Companies often want uplift in conversion rate (clicks, purchases, sign-ups).
- Minimum Detectable Uplift (MDU/MDE): Smallest uplift we want to be able to detect with given sample size, α, and power.
- Decision criteria:
- If uplift is statistically significant and positive → rollout treatment.
- If uplift is not significant → inconclusive.
- If uplift is significantly negative → stop or reconsider feature.
5. Key Takeaways
- Absolute uplift shows the raw difference in percentage points.
- Relative uplift shows proportional improvement relative to baseline.
- Both are important:
- A 1% absolute uplift may sound small, but if baseline is 2%, that’s a 50% relative improvement.
- Statistical testing (two-proportion z-test or Bayesian posterior) determines if uplift is real vs random noise.
In short:
Conversion Rate Uplift = the improvement in conversion rate of treatment over control.
- Absolute uplift = difference in percentage points.
- Relative uplift = percentage improvement relative to baseline.
It’s the main success metric in A/B testing.