A/B test sample size calculator

Plan traffic before starting an A/B test

Use this calculator to estimate how many visitors each of two variants needs to detect a specified increase in a conversion rate. It is a planning estimate for a binary outcome, such as signed up or did not sign up, with an equal traffic split.

1. Enter Baseline conversion rate % using historical data for the same audience and conversion event. For example, 5 means five conversions per 100 eligible visitors.
2. Enter the smallest improvement that would matter to your decision as Minimum detectable effect. Under Effect is, choose Relative (% of the baseline) or Absolute (percentage points).
3. Choose the Significance level and Statistical power before the experiment starts. Optionally enter Daily visitors across both variants using the total eligible traffic, not the traffic to just one arm.
4. Select Calculate sample size. Read the visitors per variant, total visitors and estimated days. Use the effect-size table to see how a larger detectable change affects the required traffic.

Example: detecting an increase from 5% to 5.5%

Baseline conversion rate %: 5
Minimum detectable effect: 10
Effect is: Relative (% of the baseline)
Daily visitors across both variants: 2000
Significance level: 5% — 95% confidence
Statistical power: 80%

These inputs return 31,234 visitors per variant, 62,468 in total and about 31.2 days. A 10% relative increase on a 5% baseline gives a 5.5% target, a difference of 0.5 percentage points. Entering 0.5 with Absolute selected describes the same target. Entering 10 as an absolute effect instead asks for a 15% target and answers a very different question.

Reading significance, power and duration

The form uses a two-sided normal approximation for two independent proportions and a 50/50 allocation. A 5% significance level sets the nominal false-positive rate under the null hypothesis; it does not mean there is a 95% probability that a winning variant is better. Power is the probability of detecting the specified effect under the model, not a guarantee that the effect exists.

Estimated days equals total required visitors divided by your daily traffic. It assumes traffic remains steady and both variants receive equal allocation. Round up when planning a run and allow for the business cycles relevant to your audience. A smaller minimum detectable effect usually needs more traffic.

Where this estimate has limits

Use independent, consistently assigned experimental units and one predefined conversion measure. Repeated page views by the same visitor are not automatically independent samples. This form does not account for unequal allocation, more than two variants, multiple metrics, clustered users, revenue averages or sequential stopping rules.

Choose the sample size and analysis plan in advance. Repeatedly checking an ordinary fixed-sample test and stopping as soon as it appears significant changes its false-positive behavior. This tool plans sample size; it does not analyze completed results or select a winner. Rare events and small samples may need a method better suited to the experiment.

What you submit

The numeric inputs are sent to the ShortFreeURL server for calculation. Aggregate rates and traffic estimates are sufficient; do not enter visitor identities or upload raw customer records. The baseline must be above 0% and below 100%, and the chosen positive effect must leave the target below 100%.

Reference

• Statistical reference: normal-approximation planning for two proportions

Useful next steps

• Inspect conversion rates across a funnel
• Plan a traffic allocation
• Label campaign traffic consistently