Two-Proportion P-Value Calculator
Compare two conversion rates with a pooled two-proportion z-test and see the difference alongside its p-value.
Why use this calculator
Turn conversion counts into evidence
A conversion rate can look better in treatment simply because of random variation. This calculator tests whether the observed difference between two independent proportions is larger than would be expected under a no-difference model.
What does a p-value mean?
The p-value is the probability of observing a result at least as extreme as this one if the two conversion rates were truly equal. It is not the probability that the treatment has no effect, and it does not tell you whether the difference is large enough to matter.
Use the p-value with the observed lift, confidence interval, pre-planned sample size, and guardrail metrics. A statistically significant conversion change can still be too small to justify a rollout, while a practically meaningful change may need more data before it is conclusive.
How to calculate a p-value for two proportions
For an A/B test with a binary outcome, the two-proportion z-test compares the conversion rate in control with the conversion rate in treatment. It pools the two rates under the null hypothesis, calculates a z statistic, and converts that statistic into a p-value for your selected alternative hypothesis.
Use assigned users as the denominator
Enter every eligible user assigned to each variant, not only people who reached a later funnel step. Changing the denominator after treatment assignment can turn a clean conversion metric into a conditional comparison and introduce bias.
Choose one-sided tests before seeing the data
A two-sided test is the usual default because it can detect either an improvement or a regression. Use a one-sided alternative only when the direction was genuinely specified before looking at the result and the opposite direction would not change the decision.