About the CI for Difference of Proportions
This calculator builds a confidence interval for the gap between two group proportions, such as two conversion rates, response rates, or approval rates measured on independent samples, the kind of comparison behind most A/B tests and two-arm surveys. Instead of a single yes/no test result, it returns a plausible range for the actual size of that gap, expressed in percentage points.
How It Works
Enter the number of successes and the total sample size for each of two groups, along with a confidence level. The calculator converts each group's counts into a proportion, subtracts the two proportions to get a point estimate of the difference, computes the standard error of that difference from both groups' individual variability, and multiplies by the z-score for the chosen confidence level to set the margin around the estimate.
Formula & Methodology
Each group contributes its own p(1-p)/n term to the variance under the square root, so a group with a proportion near 50% and a modest sample size can add as much uncertainty as a group with a more extreme proportion but a much larger sample, since p(1-p) is largest when p is near 0.5 and shrinks as a proportion approaches 0 or 1.
Examples
Email Campaign Comparison
Campaign A converts 120 of 1,000 recipients; Campaign B converts 145 of 1,000. At 95% confidence, the interval for the difference in conversion rate runs from about -5.47% to 0.47%, crossing zero, so the two campaigns aren't shown to differ significantly at this confidence level.
Two Treatment Groups
300 of 1,000 patients respond to Treatment A; 220 of 1,000 respond to Treatment B. At 99% confidence, the interval for the difference in response rate runs from about 2.97% to 13.03%, staying entirely above zero.
Advantages
- Reports the likely size of a gap between two rates, not just whether that gap clears a significance threshold.
- Weighs each group's own sample size and observed proportion when building the standard error, rather than treating both groups as equally precise.
- Makes it simple to see at a glance whether zero sits inside or outside the computed range.
Common Mistakes
- Subtracting two percentages by hand and treating that raw gap as reliable, without accounting for how much sampling uncertainty surrounds each rate.
- Assuming an interval that crosses zero proves the two groups have identical true proportions, when it really means zero can't be ruled out as a possibility.
- Applying this independent-samples formula to before and after measurements on the same group of people, which needs a paired approach instead.
Edge Cases to Watch For
- Successes cannot exceed the group's total size in either group, and both group sizes must be greater than zero, or the calculator returns an error.
- The standard error uses each group's own observed proportion (an unpooled approach), which is appropriate for estimating the size of a difference, as opposed to a pooled-proportion version used specifically for testing a null hypothesis of exactly zero difference.
- With small samples or proportions near 0% or 100%, this style of interval can technically extend past 0% or 100%, since nothing in the formula enforces that boundary.
Common Use Cases
- A/B testers comparing conversion, click-through, or signup rates between two independent variants.
- Clinical or public health analysts comparing response or recovery rates between two treatment groups.
- Pollsters and survey researchers comparing approval or agreement rates between two independent respondent groups.