The minimum detectable effect calculator above answers the question that should be asked before any conversion test starts: given this baseline rate and this much traffic, what is the smallest improvement the test could reliably identify? If the answer is a 16% relative lift and you are testing a button colour, the test is already finished — it just has not told you yet.
Arb Digital runs this check before committing client traffic to any experiment, because an underpowered test is worse than no test. It consumes weeks, produces an inconclusive result, and then gets read as evidence that the change "did not work", when in truth the design was never capable of detecting a change of that size in the first place.
What This Minimum Detectable Effect Calculator Does
Enter your baseline conversion rate and the sample size available per variant, choose your significance level and statistical power, and the tool returns the minimum detectable effect in both forms that matter: the absolute difference in percentage points and the relative lift that difference represents. It also shows the variant conversion rate you would need to observe for the result to clear your significance threshold.
The tool then works in the opposite direction. Enter the relative lift you actually hope to achieve and it calculates the sample size per variant required to detect it, and — using your daily traffic — how many days that would take. The bars show how the minimum detectable effect shrinks as sample size doubles and quadruples, which is the clearest way to see why precision is so expensive.
How to Use It
- Enter your baseline conversion rate. Use a recent, stable period of control performance rather than an all-time average that spans different traffic mixes.
- Enter the sample size available per variant for the period you can realistically run the test — this is per group, not the combined total.
- Choose significance and power. The conventional defaults are 95% significance and 80% power, both of which are conventions rather than laws.
- Enter the relative lift you hope to see to get the sample and the number of days it would need.
- Click Calculate and compare the two answers. If your target lift is smaller than the minimum detectable effect, redesign the test before launching it.
The Formula / How It's Calculated
The calculator uses the standard two-proportion formula for a two-sided test with equal group sizes. The absolute minimum detectable effect is:
δ = (zα/2 + zβ) × √(2p(1 − p) ÷ n)
where p is the baseline conversion rate as a decimal, n is the sample size per variant, zα/2 is the critical value for your significance level (1.96 at 95%), and zβ is the critical value for your power (0.84 at 80%). Relative minimum detectable effect is simply δ ÷ p.
With a 3% baseline and 20,000 per variant: √(2 × 0.03 × 0.97 ÷ 20,000) = 0.001706, multiplied by (1.96 + 0.84) = 2.80 gives δ ≈ 0.0048, or 0.48 percentage points. Relative to a 3% baseline that is a 15.9% lift — the variant would need to convert at about 3.48% before the test could call it. Rearranged for sample size, n = (zα/2 + zβ)² × 2p(1 − p) ÷ δ², which for a 10% relative lift on the same baseline requires roughly 50,800 per variant.
The assumptions matter and should be stated plainly: equal-sized groups, a two-sided test, independent observations, a stable baseline, the normal approximation to the binomial, and pooled variance under the null hypothesis. The NIST/SEMATECH e-Handbook of Statistical Methods sets out the underlying test and its conditions in full.
Absolute and Relative Lift Are Not the Same Thing
Most disagreements about test results come down to this confusion. A move from 3.0% to 3.5% is an absolute difference of 0.5 percentage points and a relative lift of about 17%. Both describe the same result, and stakeholders routinely hear one and act on the other. A stakeholder told the test can detect "half a percent" will usually assume a relative half a percent, which on a 3% baseline would need roughly three thousand times the sample.
Always say which one you mean, and prefer percentage points when describing differences in a rate. This calculator reports both deliberately: the headline is the relative lift, because that is how targets are usually set, and the grid shows the absolute difference in percentage points, because that is what the statistics actually work with.
Low Baseline Rates Make Everything Much Harder
The sample size required to detect a given relative lift rises sharply as the baseline conversion rate falls. Intuitively, a rare event carries more relative noise: at a 0.5% conversion rate, a handful of extra conversions in one group can shift the observed rate substantially, so the test needs far more data before a real difference can be separated from ordinary variation.
This is why checkout-step tests, which usually run on high baseline rates, can be resolved in days while lead-form tests on a 1% baseline take months. If your minimum detectable effect is unworkable, one of the most effective fixes is to move the test to a metric with a higher base rate — measure add-to-cart rather than purchase, or form-start rather than qualified lead — provided you have evidence the two move together. Check your current rates with the conversion rate calculator before choosing which step to test.
Power and Significance Both Cost Traffic
Tightening either threshold increases the sample you need, and the two are often confused. Significance controls false positives: the risk of declaring a winner that does not exist. Power controls false negatives: the risk of missing a real improvement. The conventional 95% and 80% settings mean you are five times more tolerant of missing a real effect than of inventing one.
That trade-off should be a business decision, not a default. If a change is cheap to implement and reversible, accepting a slightly higher false-positive risk in exchange for a faster result is often rational. If a change is expensive or hard to undo, tighten significance and accept the longer test. Switch the selectors in this calculator to see the traffic cost of each choice before you decide, and note that quadrupling the sample only halves the minimum detectable effect, because sample size appears under a square root.
Calculate It Before the Test, Not After
Minimum detectable effect is a planning tool. Calculating it after an inconclusive test to explain the result is a well-known statistical trap: once the data is in, the observed result already contains the information, and a retrospective power calculation adds nothing except a justification narrative.
The discipline that works is simple. Before launching, calculate the minimum detectable effect for the traffic you can realistically commit. If it is larger than any lift the change could plausibly produce, do not run the test — either redesign it as a bolder change, move it to a higher-traffic page, choose a metric with a higher base rate, or accept the decision will be made on judgement rather than data. Fix the sample size and the end date in advance, then stop when you reach them. If the test is running inside an ad platform rather than on your own site, the experiment features documented in Google Ads Help handle the traffic split for you, but they do not choose the duration — that is still your decision. Once the test is running, evaluate the result with the A/B test calculator and size any follow-up with the sample size calculator.
Arb Digital designs conversion tests around the traffic you actually have — bold enough changes to be detectable, metrics with workable base rates, and fixed stopping rules agreed before launch.
Web Development & CRO Paid Advertising ServicesCommon Mistakes to Avoid
- Confusing absolute and relative lift — 0.5 percentage points and a 0.5% relative improvement differ by orders of magnitude in required sample.
- Calculating power after the test — retrospective power analysis explains a result rather than informing a design.
- Using combined traffic as the sample size — the formula needs the count per variant, not the total across both.
- Stopping as soon as the result looks significant — repeatedly checking accumulating data inflates the false-positive rate well beyond the stated level.
- Testing tiny changes on low-traffic pages — if the minimum detectable effect exceeds any plausible improvement, the test cannot succeed.
Related Free Tools From Arb Digital
Size the test with the sample size calculator, evaluate the result with the A/B test calculator, and check your starting point with the conversion rate calculator or the landing page conversion calculator. Translate a confirmed lift into economics with the CPA calculator, and browse the full free online tools hub.
Frequently Asked Questions
It is the smallest difference between two variants that a test of a given size can reliably identify at your chosen significance level and power. Effects smaller than this may well be real, but the test does not have enough data to distinguish them from random variation.
It can be expressed either way and both are shown here. The absolute figure is the difference in percentage points, while the relative figure expresses that same difference as a percentage of the baseline rate.
Conventional defaults are 95% significance and 80% power, which means accepting a one-in-twenty chance of a false positive and a one-in-five chance of missing a real effect. Choose them according to the cost of each type of error in your situation rather than by habit.
Rare events carry proportionally more random variation, so more observations are needed before a genuine difference can be separated from noise. Halving the baseline rate roughly doubles the sample needed to detect the same relative lift.
Longer tests accumulate more sample, which lowers the minimum detectable effect, but only with the square root of the sample — quadrupling the duration halves it. Very long tests also risk the baseline itself shifting due to seasonality or traffic mix changes.
No. This calculation is for proportions such as conversion rate. Metrics like revenue per visitor are continuous and highly skewed, so they need a different approach based on the variance of the values rather than on a binomial proportion.
Test a bolder change, move the test to a page with more traffic, or measure a step with a higher base rate that is known to move with the outcome you care about. Running an underpowered test anyway produces an inconclusive result that is easily misread as evidence of no effect.
Results produced by this tool are statistical estimates only — they assume equal group sizes, a stable baseline, independent observations and the normal approximation to the binomial distribution.
