A/B Test Calculator: Compare Clicks & Conversions
A/B Test Calculator
Compare two rates and understand what to test next.
Your comparison
Method: two-sided pooled two-proportion z-test, with a fixed 5% significance threshold. This is a fixed-horizon comparison: choose the sample size and duration before testing. Repeatedly checking and stopping at significance invalidates the usual interpretation. Significance does not establish profitability or explain the cause.
Plan your next creative test
Describe the audience, offer and the one difference between A and B.
This generates a prompt locally; it does not send your data to an AI service.
Frequently asked questions
What does this A/B calculator compare?
It compares the proportion of trials with a successful outcome in two separate groups. For conversion tests, count unique visitors and visitors who converted. For CTR, use impressions and impressions that produced a click; statistical inference requires independent trials.
What does statistical significance mean?
A p-value below 0.05 indicates evidence against equal underlying rates under this test’s assumptions. It is not the probability that a variation is the winner, and it does not prove a business benefit.
Why is my result inconclusive?
The observed difference may be too small relative to the sampling uncertainty, or the counts may be too sparse for this approximation. An inconclusive result does not prove the variations are equivalent.
How long should I run a test?
Plan the duration and sample size around your baseline rate and the smallest improvement worth detecting. Include relevant business cycles. There is no universal seven-day or 1,000-impression rule.
What should I test first?
Choose a specific hypothesis: a clearer headline, a different product image, a more relevant offer or a clearer call to action. Keep other conditions comparable and use purchases or qualified leads when those are your goal.
Method reference: NIST comparison of two proportions
Educational calculations only. Check your experiment design and business outcomes before acting on a result.