Free A/B Test Significance Calculator
Find out if your split test result is statistically significant in seconds. Plug in visitors and conversions for both variants to see the winner, the lift, the p-value, and the sample size you need.
Confidence level
Confidence
0.0%
Waiting for dataEnter visitors and conversions for both variants to see your verdict.
- Control (A) conversion rate
- 0.00%
- Variant (B) conversion rate
- 0.00%
- Lift
- --
- p-value
- --
- Sample size
- --
Pro tip. Do not stop a test the moment confidence crosses 95%. Decide on your sample size up front and let the test run. Otherwise you are likely calling false positives.
How it works
How to check A/B test significance in 3 steps
Get a clear, statistically sound verdict on your split test in under a minute, with no spreadsheets and no signup.
- 01
Enter your control numbers
Add the total visitors and conversions for variant A, the original version you are testing against.
- 02
Enter your variant numbers
Add visitors and conversions for variant B, the new version. Your conversion rates update on the fly.
- 03
Read your verdict
See whether the result is significant, the lift, p-value and confidence, plus an estimate of how many more visitors you need if you are not there yet.
Why it matters
Why marketers run a significance check
A higher conversion rate alone does not mean a winner. Use a proper significance test to know when to stop, when to keep running, and when a result is just noise.
Know when to stop a test
Call winners with real confidence instead of guessing, and avoid leaving working variants on the table.
Avoid false positives
Stop the peeking problem from costing you money. A proper p-value tells you when a difference is real.
Plan your next test
The sample-size hint estimates how many visitors each variant needs, at 80% power, to confirm the difference you're seeing. Use it to size your next test before it starts.
Cleaner reporting
Share the result link with your team in one click so everyone reads the same numbers, with the same confidence.
Use cases
Where teams use the A/B test calculator
From landing pages to ad creatives, here is how marketers put a quick significance check to work across every experiment they run.
Landing page tests
Compare two versions of a landing page to know which headline, hero or CTA actually moves the conversion rate.
Facebook & Meta Ads
Decide between two ad creatives or audiences with real statistical backing instead of trusting Ads Manager labels.
Google Ads copy tests
Compare two responsive search ad variants to confirm a new headline really beats the control before scaling spend.
Email subject line tests
Send two subject lines to a small slice of your list and call the winner with confidence before the full broadcast.
Checkout flow tests
Validate that a new checkout step, trust badge or upsell really improves completion rate and is not noise.
Pricing page experiments
Run a clean test between two pricing tiers or layouts and decide based on signups, not on gut feel.
FAQ
Questions about the A/B Test Significance Calculator.
Anything not here is in the Help Center, or ask us.
FAQs
What is statistical significance in A/B testing?
Statistical significance is a measure of how confident you can be that the difference between two variants is real and not just random variation. A common rule of thumb is to call a winner when confidence is 95% or higher, meaning there is less than a 5% chance the result happened by luck.
What is a p-value?
A p-value is the probability of seeing your observed result (or something more extreme) if there was actually no real difference between the variants. The smaller the p-value, the stronger the evidence that the difference is real. A p-value of 0.05 corresponds to 95% confidence.
What confidence level should I use?
Most marketers use 95% confidence, which means you accept a 5% chance of a false positive. Use 99% if the decision is high-stakes (for example, rolling out a change to all paid traffic) and 90% only for low-risk early-stage experiments.
How many visitors do I need for an A/B test?
It depends on your baseline conversion rate and the size of the lift you want to detect. As a rough rule, small lifts (under 5%) need tens of thousands of visitors per variant, while bigger lifts (20% and up) can be confirmed with a few thousand. The calculator estimates how many more visitors each variant needs to confirm the difference you're seeing, at 80% power.
What is lift in an A/B test?
Lift is the relative percentage difference between the two variants' conversion rates. If variant A converts at 3% and variant B converts at 4.5%, the lift of B over A is 50%. Lift is what you actually use to decide if a change is worth shipping.
Why is my test not significant even though variant B is winning?
A higher conversion rate alone is not enough. You need enough data for the difference to be unlikely to come from random chance. The smaller the lift, the more visitors you need. If you stop a test too early, you risk shipping a change that was just noise.
What is the peeking problem?
Peeking is checking your test results repeatedly and stopping the moment confidence crosses 95%. Doing this dramatically increases your false positive rate. To avoid it, decide on the sample size you need up front, then check the result once at the end.
Does this calculator work for Facebook, Google or TikTok Ads?
Yes. Significance is a universal statistic, so the calculator works for any A/B test where you can measure visitors (or impressions, clicks, sends) and conversions. Plug in numbers from Meta Ads, Google Ads, TikTok Ads, email platforms or your landing page analytics.
Is this A/B test calculator free?
Yes. The A/B test significance calculator is 100% free, with no signup, no limits and no email gate. Run as many tests as you need for personal or commercial use.
Is my data stored anywhere?
No. All calculations happen entirely in your browser. Visitor counts and conversion numbers never leave your device and are never sent to our servers.
A/B test the landing page your visitors actually see
The biggest A/B test wins come from testing the page where your paid traffic lands. Use LanderLab to build, A/B test and ship landing pages that convert, then bring the numbers back to this calculator.