2014–15 MacroMonitor Sampling Tolerance Tables

Sampling Tolerance for Individual Percentages

Sampling Tolerance for Individual Percentages
(95% Confidence Level)

  Approximate Sampling Tolerance for a Survey Percentage at or near These Levels
Size of Sample
on Which Survey
Result Is Based
10% or 90% 20% or 80% 30% or 70% 40% or 60% 50%
4400 2% 3% 4% 4% 4%
3500 3 3 4 5 5
2000 4 4 6 6 6
1500 4 5 7 7 7
1000 5 6 8 9 9
750 6 7 9 10 10
500 7 9 11 12 12
250 10 14 16 17 18
100 16 20 26 27 28

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages in the same sample. Look at this chart for the column closest to the two percentages. Look for the row that is closest to the unweighted sample size. At the location where they cross is a number that must be smaller than the gap between the two percentages. For example, if you were looking at a question for which 32% of the respondents had selected one answer and 28% had selected another and for which the sample consisted of 1,100 unweighted respondents, you would look at where the 30% column intersected with the 1,000 sample size and find 8. Because the gap between 25% and 34% is greater than 8, these two numbers are statistically significant. Put another way: Significantly more respondents selected the 34% answer than the 25% answer.


Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 10% and 90%

Table for Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 10% and 90%

Sample Size 4000 3500 2000 1500 1000 750 500 250 100
4000 1.9% 2.0% 2.3% 2.6% 3.0% 3.4% 4.0% 5.6% 8.6%
3500 2.0 2.0 2.4 2.6 3.1 3.4 4.1 5.6 8.7
2000 2.3 2.4 2.7 2.9 3.3 3.6 4.3 5.7 8.7
1500 2.6 2.6 2.9 3.1 3.5 3.8 4.4 5.8 8.8
1000 3.0 3.1 3.3 3.5 3.8 4.1 4.7 6.0 8.9
750 3.4 3.4 3.6 3.8 4.1 4.4 4.9 6.2 9.1
500 4.0 4.1 4.3 4.4 4.7 4.9 5.4 6.6 9.3
250 5.6 5.6 5.7 5.8 6.0 6.2 6.6 7.6 10.0
100 8.6 8.7 8.7 8.8 8.9 9.1 9.3 10.0 12.1

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages that are close to 10% or 90% of different subsamples. First, determine the unweighted sample sizes for the two populations. Next, look at the above chart for the intersection that comes closest to the two unweighted sample sizes. Finally, if the gap between the two percentages that you have is larger than the number in the intersection, then the two percentages are statistically different at the 95% confidence interval. For example, suppose the two percentages that you are comparing are 8% and 12% and the unweighted sample sizes are 450 and 705, respectively. Looking at the table above, the closest intersection of these numbers occurs at the 500 and 750 sample sizes. The number in this cell is 4.9. The gap between the two percentages is 4. Therefore, the difference is not statistically significant at the 95% confidence interval.


Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 20% and 80%

Table for Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 20% and 80%

Sample Size 4000 3500 2000 1500 1000 750 500 250 100
4000 2.5% 2.6% 3.1% 3.4% 4.0% 4.5% 5.4% 7.4% 11.5%
3500 2.6 2.7 3.2 3.5 4.1 4.6 5.4 7.4 11.5
2000 3.1 3.2 3.6 3.9 4.4 4.9 5.7 7.6 11.6
1500 3.4 3.5 3.9 4.2 4.6 5.1 5.9 7.8 11.7
1000 4.0 4.1 4.4 4.6 5.1 5.5 6.2 8.0 11.9
750 4.5 4.6 4.9 5.1 5.5 5.9 6.6 8.3 12.1
500 5.4 5.4 5.7 5.9 6.2 6.6 7.2 8.8 12.5
250 7.4 7.4 7.6 7.8 8.0 8.3 8.8 10.2 13.5
100 11.5 11.5 11.6 11.7 11.9 12.1 12.5 13.5 16.1

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages that are close to 20% or 80% of different subsamples. First, determine the unweighted sample sizes for the two populations. Next, look at the above chart for the intersection that comes closest to the two unweighted sample sizes. Finally, if the gap between the two percentages that you have is larger than the number in the intersection, then the two percentages are statistically different at the 95% confidence interval. For example, suppose the two percentages that you are comparing are 18% and 26% and the unweighted sample sizes are 431 and 728, respectively. Looking at the table above, the closest intersection of these numbers occurs at the 500 and 750 sample sizes. The number in this cell is 6.6. The gap between the two percentages is 8. Therefore, the difference is statistically significant at the 95% confidence interval.


Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 30% and 70%

Table for Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 30% and 70%

Sample Size 4000 3500 2000 1500 1000 750 500 250 100
4000 2.9% 3.0% 3.6% 3.9% 4.6% 5.2% 6.2% 8.5% 13.2%
3500 3.0 3.1 3.6 4.0 4.7 5.2 6.2 8.5 13.2
2000 3.6 3.6 4.1 4.5 5.0 5.6 6.5 8.7 13.4
1500 3.9 4.0 4.5 4.8 5.3 5.8 6.7 8.9 13.5
1000 4.6 4.7 5.0 5.3 5.8 6.3 7.1 9.2 13.7
750 5.2 5.2 5.6 5.8 6.3 6.7 7.5 9.5 13.9
500 6.2 6.2 6.5 6.7 7.1 7.5 8.2 10.1 14.3
250 8.5 8.5 8.7 8.9 9.2 9.5 10.1 11.6 15.4
100 13.2 13.2 13.4 13.5 13.7 13.9 14.3 15.4 18.4

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages that are close to 30% or 70% of different subsamples. First, determine the unweighted sample sizes for the two populations. Next, look at the above chart for the intersection that comes closest to the two unweighted sample sizes. Finally, if the gap between the two percentages that you have is larger than the number in the intersection, then the two percentages are statistically different at the 95% confidence interval. For example, suppose the two percentages that you are comparing are 68% and 72% and the unweighted sample sizes are 1,431 and 728, respectively. Looking at the table above, the closest intersection of these numbers occurs at the 1,500 and 750 sample sizes. The number in this cell is 5.8. The gap between the two percentages is 4. Therefore, the difference is not statistically significant at the 95% confidence interval.


Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 40% and 60%

Table for Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 40% and 60%

Sample Size 4000 3500 2000 1500 1000 750 500 250 100
4000 3.1% 3.2% 3.8% 4.2% 4.9% 5.5% 6.6% 9.1% 14.1%
3500 3.2 3.3 3.9 4.3 5.0 5.6 6.7 9.1 14.1
2000 3.8 3.9 4.4 4.8 5.4 6.0 7.0 9.3 14.3
1500 4.2 4.3 4.8 5.1 5.7 6.2 7.2 9.5 14.4
1000 4.9 5.0 5.4 5.7 6.2 6.7 7.6 9.8 14.6
750 5.5 5.6 6.0 6.2 6.7 7.2 8.0 10.2 14.8
500 6.6 6.7 7.0 7.2 7.6 8.0 8.8 10.8 15.2
250 9.1 9.1 9.3 9.5 9.8 10.2 10.8 12.4 16.5
100 14.1 14.1 14.3 14.4 14.6 14.8 15.2 16.5 19.7

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages that are close to 40% or 60% of different subsamples. First, determine the unweighted sample sizes for the two populations. Next, look at the above chart for the intersection that comes closest to the two unweighted sample sizes. Finally, if the gap between the two percentages that you have is larger than the number in the intersection, then the two percentages are statistically different at the 95% confidence interval. For example, suppose the two percentages that you are comparing are 38% and 44% and the unweighted sample sizes are 950 and 3,450 respectively. Looking at the table above, the closest intersection of these numbers occurs at the 1,000 and 3,500 sample sizes. The number in this cell is 5. The gap between the two percentages is 6. Therefore, the difference is statistically significant at the 95% confidence interval.


Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages around 50%

Table for Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages around 50%

Sample Size 4000 3500 2000 1500 1000 750 500 250 100
4000 3.2% 3.3% 3.9% 4.3% 5.0% 5.6% 6.7% 9.3% 14.4%
3500 3.3 3.4 4.0 4.4 5.1 5.7 6.8 9.3 14.4
2000 3.9 4.0 4.5 4.8 5.5 6.1 7.1 9.5 14.6
1500 4.3 4.4 4.8 5.2 5.8 6.4 7.3 9.7 14.7
1000 5.0 5.1 5.5 5.8 6.4 6.9 7.8 10.0 14.9
750 5.6 5.7 6.1 6.4 6.9 7.3 8.2 10.4 15.1
500 6.7 6.8 7.1 7.3 7.8 8.2 9.0 11.0 15.6
250 9.3 9.3 9.5 9.7 10.0 10.4 11.0 12.7 16.8
100 14.4 14.4 14.6 14.7 14.9 15.1 15.6 16.8 20.1

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages that are close to 50% of different subsamples. First, determine the unweighted sample sizes for the two populations. Next, look at the above chart for the intersection that comes closest to the two unweighted sample sizes. Finally, if the gap between the two percentages that you have is larger than the number in the intersection, then the two percentages are statistically different at the 95% confidence interval. For example, suppose the two percentages that you are comparing are 48% and 53% and the unweighted sample sizes are 1,031 and 528, respectively. Looking at the table above, the closest intersection of these numbers occurs at the 1,000 and 500 sample sizes. The number in this cell is 7.8. The gap between the two percentages is 5. Therefore, the difference is not statistically significant at the 95% confidence interval.