Control vs Variant refers to the two groups in an . The control is your existing version. The variant (also called treatment or challenger) is the modified version you're testing against it.
What is Control?
The control is your baseline. It's the current version of a page, feature, or element that you're already using. In testing terminology, it represents the "no change" scenario.
Control characteristics:
- Current live version
- Represents the status quo
- Used as the benchmark for comparison
- Sometimes called "A" in
What is a Variant?
The variant is your hypothesis in action. It's the modified version that incorporates the change you want to test.
Variant characteristics:
- Contains the proposed change
- Tests a specific hypothesis
- Sometimes called "B" (or C, D for multiple variants)
- Also known as treatment, challenger, or test version
How Traffic Splits Work
Traffic divides randomly between control and variant. Common splits:
50/50 split: Equal traffic to each version. Standard for most tests.
90/10 or 80/20 split: Lower risk when testing significant changes. Less traffic to variant limits exposure if it performs poorly.
Multi-variant splits: Testing multiple variants (A/B/C/D) divides traffic further. Each variant needs enough traffic for . Before running a test, calculate how much traffic you'll need using an A/B test sample size calculator to ensure your results will be statistically valid.
Isolation Principle
The only difference between control and variant should be the element you're testing. If multiple things change, you can't attribute results to any single change.
Good test: Control has blue button, variant has green button. One variable.
Bad test: Variant has green button, different headline, and new layout. Multiple variables make results inconclusive.
Common Mistakes
-
Changing the control mid-test: Keep control stable throughout the test duration.
-
Too many variants: Each variant needs sufficient traffic. More variants means longer tests or less reliable results.
-
Variant too similar to control: Minor changes may not produce detectable differences. Test meaningful hypotheses.
