A/B Test Significance Calculator
Find out whether your A/B test result is real or just noise. Enter visitors and conversions for your control and variant to get the conversion rates, relative uplift, z-score, p-value, and a clear significance verdict, computed live in your browser.
Variant A (control)
Variant B
Uplift (B vs A)
+30.0%
p-value
0.0355
z-score
2.10
Power
56%
Variant B converts 30.0% higher than A. This is statistically significant at 95% confidence (p = 0.0355).
Email me this result + a creative-testing checklist.
How to design A/B tests that reach significance faster.
How significance works
This calculator runs a two-proportion z-test. It pools both variants' conversion rates, measures how far apart they are in standard errors (the z-score), and converts that into a p-value - the probability you would see a gap this large if the variants were truly identical. When the p-value drops below your threshold (5% at 95% confidence), the result is significant.
How many conversions do I need?
There is no fixed number - it depends on your baseline conversion rate and how big the difference is. A large uplift (say 10% to 15%) reaches significance with far less traffic than a small one (10% to 10.5%). The calculator estimates the additional samples per variant you need, so you know whether to keep the test running or call it.
P-value, explained simply
The p-value answers one question: if the two variants actually performed the same, how likely is a difference this big by pure chance? A p-value of 0.04 means a 4% chance, so at the 95% bar (5%) it counts as significant. It is not the probability that B is better - it is the probability the gap is a fluke.
The quick answer
Statistical significance tells you whether the difference between two variants is real or just random chance. In A/B testing the standard bar is 95% confidence, meaning there is less than a 5% probability the result is a fluke. Below that, keep the test running before you call a winner.
Frequently asked questions
What is statistical significance in A/B testing?
It is the probability that the difference you see between two variants is real rather than random chance. A result is "significant" when the p-value is below your threshold, usually 0.05 (95% confidence). Until then, the difference could be noise.
How many conversions do I need for a significant result?
It depends on your baseline rate and the size of the difference. Smaller uplifts need far more traffic. This calculator estimates the additional samples per variant you need to reach significance at your chosen confidence level.
What is a p-value, in plain English?
The p-value is the chance you would see a difference this large if the two variants were actually identical. A p-value of 0.03 means a 3% chance the result is luck, so at 95% confidence (a 5% bar) it counts as significant.
One-tailed or two-tailed - which should I use?
Use two-tailed (the default) when you just want to know if the variants differ in either direction. Use one-tailed only when you specifically predicted the variant would beat the control and you do not care about the other direction.
What confidence level should I use?
95% is the standard for most marketing A/B tests. Use 90% for faster, lower-stakes reads and 99% when a wrong call would be expensive. Higher confidence requires more data to reach significance.
Keep exploring
Generate the creative
Stop guessing which creative wins.
Generate and test multiple ad variants fast in Salk AI, so you reach a significant winner sooner instead of waiting on underpowered tests.
Open Salk Studio