Calculateus

A/B Test Sample Size Calculator

Calculate the sample size needed per variant to reliably detect a minimum effect size in an A/B test.

Result

Sample Size per Variant
31,232
Target Conversion Rate
5.5%

Assumes a two-sided test at 95% confidence (α = 0.05) - larger minimum detectable effects need smaller samples, while detecting subtle changes requires much larger samples.

About the A/B Test Sample Size

This calculator determines how many users or observations each variant of an A/B test needs before the test can reliably detect a chosen minimum effect. It takes a baseline conversion rate, the smallest relative change worth detecting, and a desired statistical power, then returns the required sample size per group. It is built for planning experiments before launch, not for analyzing results after the fact.

How It Works

You provide the current baseline conversion rate, the minimum relative lift you want the test to be able to detect, for example 10% means detecting a move from 5% to 5.5%, and a statistical power level of either 80% or 90%. The calculator converts the target power into a z-score, fixes the significance level at a two-sided 95% confidence with z=1.96, and combines these with the variance of both the baseline and target conversion rates to compute the minimum sample size per variant. The result is rounded up to the nearest whole visitor or observation, since a fractional sample size cannot be recruited.

n = (z_alpha + z_beta)^2 x [p1(1-p1) + p2(1-p2)] / (p2 - p1)^2, where p1 is the baseline rate, p2 = p1 x (1 + MDE) is the target rate, z_alpha = 1.96 for two-sided 95% confidence, and z_beta is 0.8416 for 80% power or 1.2816 for 90% power.

Examples

Standard 10% lift at 80% power

With the defaults, a 5% baseline, a 10% relative minimum detectable effect, and 80% power, the target conversion rate is 5.5% and the calculator returns a required sample size of roughly 31,232 visitors per variant.

Larger effect at 90% power

For a 20% baseline conversion rate, a 15% relative minimum detectable effect with a 23% target rate, and 90% power, the required sample size drops to about 3,936 per variant, since a larger effect and higher baseline rate are both easier to detect with fewer observations.

Advantages

  • Converts a business question, how big a change matters, directly into a concrete recruitment or traffic target before an experiment starts.
  • Prevents underpowered tests that run out of data before they can distinguish a real effect from noise.
  • Shows the resulting target conversion rate alongside the sample size, so the assumed effect stays visible, not just the output number.

Common Mistakes

  • Treating the minimum detectable effect as an absolute percentage-point change rather than a relative one; a 10% MDE on a 5% baseline means detecting a move to 5.5%, not to 15%.
  • Ending an experiment early because results look flat, without checking whether the planned sample size was actually reached yet.
  • Underestimating how much smaller detectable effects inflate the required sample size, since the sample size grows roughly with the square of how small the effect is.

Edge Cases to Watch For

  • The baseline conversion rate must be strictly between 0% and 100%; a rate of exactly 0% or 100% makes the variance term meaningless.
  • The minimum detectable effect can't be 0%, since that would make the denominator, the gap between the two rates, zero and leave the required sample size undefined.
  • The significance level is fixed at a two-sided 95% confidence test with z=1.96 and can't be changed in this calculator; only the power level, 80% or 90%, is adjustable.
  • The reported figure is the sample size needed per variant, not the combined total across both groups.

Common Use Cases

  • Product managers and growth teams planning how long a conversion rate experiment needs to run before it launches.
  • UX researchers and marketers sizing landing page or pricing tests against realistic traffic volumes.
  • Data teams setting stopping rules in advance so experiments aren't peeked at and stopped prematurely.
Written & fact-checked by the Calculateus TeamLast updated August 5, 2026How we verify our formulas

Frequently asked questions

Why does a smaller minimum detectable effect require a much larger sample?

Distinguishing a small, subtle difference from random noise requires much more data than distinguishing a large, obvious one - the required sample size grows roughly with the inverse square of the effect size, so cutting the effect you want to detect in half roughly quadruples the sample size needed.

Related calculators

Confidence Interval CalculatorCalculate the confidence interval for a sample mean.Sample Size CalculatorCalculate the minimum sample size needed for a survey or study at a given confidence level and margin of error.Margin of Error CalculatorCalculate the margin of error for a survey result given sample size and confidence level.Correlation Coefficient CalculatorCalculate the Pearson correlation coefficient between two paired sets of numbers.

Conclusion

Sample size planning is what separates a conclusive A/B test from one that simply runs out of time. This calculator turns the baseline rate, desired sensitivity, and power level into a single actionable number before any traffic is spent.