Step-by-Step Guide

How to Calculate A/B Test Sample Size

Follow these 5 steps to calculate the sample size you need before running an A/B test. Includes formulas, defaults, and tips for product managers.

Last updated: April 2026

1
Estimate your baseline conversion rate

The baseline is the current conversion rate for the population you plan to test. Required sample size depends on this rate and the absolute change you want to detect.

›Pull at least four weeks of historical conversion data.
›Use the same definition you’ll use during the test.
›Match the user segment that will see the test, not your global rate.
›Use the most recent stable period if rates have been moving.

Formula

Baseline conversion rate = converters / users in the qualifying period

Pro tip: Beware of seasonality. A baseline pulled from Black Friday week will overstate normal performance.

2
Decide your minimum detectable effect (MDE)

MDE is the smallest improvement you want the test to be able to detect. Set it before the test starts. A smaller MDE needs a much bigger sample.

›1-2% relative lift is hard to detect even on high-traffic sites.
›Compare several planning values, such as 5% relative lift, against your traffic and the improvement worth shipping.
›A 10-15% relative MDE requires less traffic than a smaller lift at the same baseline, but may miss worthwhile smaller improvements.
›Set MDE based on the smallest improvement worth shipping for, not your hopes.

Formula

MDE = relative percentage change you want the test powered to detect

Pro tip: Choose MDE from the decision you need to make. A larger MDE reduces sample requirements but lowers your chance of detecting smaller effects.

3
Pick your significance and power thresholds

A 5% significance level and 80% power are common planning choices. Choose them before the test based on the costs of false positives and missed improvements.

›Alpha = 0.05: a 5% false-positive rate when there is no real difference, under the planned test assumptions.
›Beta = 0.20, so power = 0.80 (80% chance of detecting a real effect at the MDE).
›Power of 80% means the planned test has an 80% chance of detecting an effect of the specified size.

Formula

Alpha = 0.05, Beta = 0.20, Power = 1 - Beta = 0.80

Pro tip: Raising alpha from 0.05 to 0.10 permits a higher false-positive rate. Do not change the threshold after seeing the results.

4
Apply the formula or use the planning calculator

Sample-size calculations combine baseline, MDE, significance, and power. The approximation below assumes a two-sided test, 95% confidence, 80% power, and equal allocation between two variants. Calculators can use more detailed approximations.

›Plug your baseline, MDE, alpha, and beta into a sample-size calculator.
›The output is required sample size per variant.
›For a two-variant 50/50 test, double the per-variant sample. Tests with more variants need planning that accounts for the comparisons you will make.
›Always round up.

Formula

N per variant ≈ 16 x p x (1 - p) / (p x relative MDE)^2; use decimals

Pro tip: Cutting off a few hundred users to make the number fit silently lowers your power. Always round up.

5
Convert sample size into a test duration

Sample size tells you how many users you need. Duration tells you how long that will take at your traffic.

›Pull your daily eligible traffic for the test surface.
›Divide required sample size by daily traffic to get days needed.
›Add a buffer to cover at least one full business cycle (minimum 7 days).
›Example: 62,000 users at 4,000 per day = roughly 16 days.

Formula

Test duration in days = total required sample size / eligible daily users

Pro tip: For a fixed-sample test, wait for the planned sample and duration. If you need valid early stopping, use a method designed for repeated checks.

Plan Your A/B Test in Seconds

Skip the manual math. Our free sample-size calculator ships with defaults for alpha, power, and MDE, plus a duration estimator.

Open Free Sample Size Calculator

Frequently Asked Questions

What's the difference between absolute and relative MDE?

Absolute MDE is a percentage-point change. Relative MDE is a percentage of the baseline. A baseline of 5% with a 10% relative MDE means you can detect a lift to 5.5%. The same 10% as absolute MDE would mean detecting a lift to 15%, which is a much bigger swing. Most sample-size calculators default to relative MDE because it's how product teams actually think about lift.

Can I run an A/B test without calculating sample size?

Plan the sample size before a fixed-sample test so you know whether available traffic can detect an improvement worth acting on. Other designs, such as sequential tests, need their own stopping rules.

What if I don't have enough traffic to hit the required sample size?

Consider a larger MDE only if detecting smaller effects is unnecessary, a longer test if conditions can stay stable, or a higher-traffic surface relevant to the same question. Make trade-offs before launch.

Does the formula work for non-binary metrics?

The formula above is for binary outcomes such as conversion. Continuous metrics such as revenue per user need a calculation that uses their variability. PM Toolkit’s sample-size calculator plans binary-outcome tests.

Do I need to recalculate if my baseline changes during the test?

No. Lock the design at test start. If the baseline shifts dramatically, that's a signal to rerun the test under the new conditions, not to adjust mid-flight.

Back to Sample Size Calculator