A/B Test Sample Size Calculator

Sample size is the number of visitors you need in each variation to get statistically reliable results. Too few visitors means a result might come from random chance rather than a real difference between versions. Calculating sample size before you start keeps the test honest and prevents false positives.

Yes. This calculator uses standard frequentist math (a two-proportion z-test), the same family of statistics behind most planning calculators:

• Adobe Target uses a Welch's t-test and recommends 95% significance with 80% power, the defaults here
• Optimizely is frequentist and expects you to set sample size upfront
• VWO's public calculator is frequentist too, though VWO SmartStats switches to Bayesian inside the product

Plan your test here, then run it in your platform of choice. The inputs (baseline conversion rate, minimum detectable effect, significance, power) map directly to what each tool asks for.

This is a frequentist calculator. It uses a two-proportion z-test to compute how many visitors each variation needs before you can trust the result.

• Frequentist (this tool, Optimizely, Adobe Target): decide sample size upfront, run to completion, no peeking
• Bayesian (VWO SmartStats, some Statsig setups): reports a probability that one variant beats another and allows continuous monitoring

Frequentist planning is the safest default because it protects against the inflated false-positive rate you get from checking results early.

This calculator uses the two-sample proportion test (two-tailed):

n = (Zα/2 × √(2p̄q̄) + Zβ × √(p1q1 + p2q2))² / (p2 − p1)²

Where:

• p1 = baseline conversion rate, p2 = expected rate after improvement
• Zα/2 = z-score for your significance level (1.96 for 95%)
• Zβ = z-score for your power (0.84 for 80%)
• p̄ = pooled probability (p1 + p2) / 2, q = 1 − p

You don't need to compute this by hand. Enter your numbers above and the calculator returns sample size per variation plus total traffic needed.

Run for at least 7 full days to capture weekly patterns, since weekdays and weekends convert differently. Use the duration estimate above to see the exact run time at your traffic level. Avoid ending a test early just because one version looks like it is winning, since early leads often disappear.

They run at different stages:

• Sample size calculator (this tool): used before the test to plan how many visitors you need
• Significance or confidence calculator: used after the test, with your final conversion counts, to check whether the difference is statistically significant

Plan with this tool first, run the test to the target sample size, then confirm the result in your testing platform's report.

MDE is the smallest relative improvement worth detecting:

• 10-20% MDE: practical for small-to-medium stores
• 5-10% MDE: workable for high-traffic stores
• Under 5% MDE: needs very high traffic volumes

The smaller the effect you want to catch, the more traffic you need.

If the required duration is too long, you have options:

• Increase MDE (aim for 20% instead of 10%)
• Lower significance to 90% for low-risk tests
• Test bigger changes instead of minor tweaks
• Focus on high-traffic pages like the homepage or product pages
• Test micro-conversions like add-to-cart instead of purchases

The Shopify conversion rate optimization guide covers high-impact changes worth testing first.

Both control different errors:

• Statistical significance (95%): the probability that a detected effect is real, not a false positive
• Statistical power (80%): the probability of detecting a real effect when it exists, avoiding false negatives

You want to find real effects (power) while avoiding false alarms (significance).

Yes, but each extra variation raises the required sample size. With 3 variations you split traffic three ways, so total visitors needed climbs by roughly 50%. Limiting to 2-3 variations works best for most stores.

95% is the working standard. Use 99% only for mission-critical changes like the checkout flow, when a wrong decision is expensive, or in enterprise environments with strict requirements. 90% significance is acceptable for low-risk tests when you need faster results.