About the Two-Proportion Z-Test
The Two-Proportion Z-Test Calculator computes the z-statistic for comparing two independent sample proportions, the standard calculation behind A/B testing significance checks. It is built for comparing conversion rates, response rates, or any other percentage-based outcome measured across two separate groups.
How It Works
Enter the number of successes and total size for each group. The calculator computes each group's proportion, pools both groups' successes and sizes into a single combined proportion, uses that pooled proportion to calculate the standard error of the difference, and divides the observed difference in proportions by that standard error to produce the z-statistic.
Formula & Methodology
To compute this by hand, find each group's proportion by dividing its successes by its size. Combine both groups' totals into a single pooled proportion, since under the null hypothesis both groups are assumed to share the same true rate. Use that pooled proportion to compute the standard error, then divide the raw difference between the two sample proportions by that standard error to get the z-statistic, which follows a standard normal distribution under the null hypothesis.
Examples
Website Signup Test
Group 1 has 120 successes out of 1000, Group 2 has 145 out of 1000. That gives p1 = 12.0%, p2 = 14.5%, a pooled proportion of 13.25%, and a z-statistic of roughly -1.72, short of the 1.96 threshold for significance.
Email Campaign Comparison
Group 1 has 300 opens out of 2000 sent, Group 2 has 360 opens out of 2000 sent. That gives p1 = 15%, p2 = 18%, a pooled proportion of 16.5%, and a z-statistic beyond negative 1.96, indicating a statistically significant difference at the 95 percent level.
Advantages
- Automatically pools the two groups' proportions before computing the standard error, matching standard significance-testing practice.
- Flags the plus or minus 1.96 significance threshold directly against your computed z-statistic for the common 95 percent case.
- Reports both individual group proportions alongside the z-statistic, so the raw difference is visible next to its statistical significance.
Common Mistakes
- Comparing raw percentage-point differences between two groups without checking whether the sample sizes are large enough for the difference to be statistically meaningful.
- Treating a significant z-statistic as proof of a large practical effect, when a tiny but real difference can still be significant with large enough sample sizes.
- Running this test repeatedly on the same ongoing experiment and stopping as soon as significance appears, which inflates the true false-positive rate.
Edge Cases to Watch For
- Both group sizes must be greater than zero, and the calculator returns an error if the computed standard error is exactly zero, which would otherwise produce an undefined result.
- The calculator flags a z-statistic beyond plus or minus 1.96 as corresponding to statistical significance at the common 95 percent confidence level for a two-tailed test.
- The standard error is built from a pooled proportion, so this test assumes both groups share the same true rate under the null hypothesis, which differs from a confidence-interval approach that estimates the two rates separately.
Common Use Cases
- Marketers and product teams evaluating whether an A/B test's variant produced a real change in conversion rate.
- Researchers comparing response or success rates between two independent groups in a survey or experiment.
- Analysts checking whether a policy or process change measurably shifted a binary outcome rate between two segments.