Conversion Lift measures how much better a test variant performs compared to your control. It expresses the improvement as a percentage, making it easy to understand the impact of changes you test.
What is Conversion Lift?
Conversion lift shows the relative improvement between two versions. If your control converts at 2% and your variant converts at 2.5%, the conversion lift is 25%.
Conversion Lift = ((Variant CR - Control CR) / Control CR) x 100
Example calculation:
- Control : 2.0%
- Variant conversion rate: 2.5%
- Lift = ((2.5 - 2.0) / 2.0) x 100 = 25%
This means the variant performs 25% better than the control.
Lift vs Absolute Change
Conversion lift (relative change) and absolute change tell different stories.
Relative lift: 25% improvement sounds significant. Good for communicating impact to stakeholders.
Absolute change: 0.5 percentage points (from 2.0% to 2.5%). Shows the actual difference in conversion rates.
Both matter. A 25% lift from 2% to 2.5% is meaningful. A 25% lift from 0.04% to 0.05% is negligible in real terms.
When Lift is Meaningful
matters. A 25% lift means nothing if your test lacks enough data. Always check confidence levels before trusting lift numbers.
Sample size affects reliability. Small samples produce unreliable lift estimates. Wait for adequate data before declaring results.
Segment carefully. Overall lift may hide segment differences. Mobile users might show 40% lift while desktop shows -5%.
Common Mistakes
-
Celebrating early: Lift fluctuates during tests. Early results often don't hold.
-
Ignoring baseline: A 50% lift on a 0.1% baseline adds 0.05 percentage points. Calculate actual impact.
-
Forgetting confidence intervals: Report lift with its . "15-35% lift" is more honest than "25% lift."
-
Mixing segments: Blended lift can be misleading. Break down by device, traffic source, and user type.
