Polling Margin of Error & Sampling: Quick Notes

Key Concepts

• Sampling error vs non-sampling error: sampling error arises from observing a sample; non-sampling errors include nonresponse bias, response bias, recall bias, selection bias, etc.
• Poll data typically uses categorical data (e.g., Labor vs National) to infer population behavior.
• The “margin of error” (MOE) quantifies the sampling error for a reported percentage.

Margin of Error and Sampling Error

• MOE (in percentage points) ≈
  $\text{MOE} \approx \frac{100}{\sqrt{n}}$
  for percentages in the 30\% to 70\% range (i.e., when p is not too close to 0 or 100).
• Examples:
  • n = 400 ⇒ MOE ≈ 100/\sqrt{400} = 5\%.
  • n = 1000 ⇒ MOE ≈ 100/\sqrt{1000} ≈ 3.16\% (about 3.2\%).
• For minor parties near 0% or 100%, MOE is smaller or the simple rule is less accurate; the exact formula would yield different values.
• The MOE reflects only sampling error, not non-sampling errors.

Confidence Intervals

• A 95\% confidence interval for a single percentage estimate is:
  $\text{Estimate} \pm \text{MOE}$
• Interpretation: If we repeated the poll many times, about 95\% of such intervals would contain the true population value.
• The interval accounts for sampling variability; true error also includes non-sampling errors (not captured here).

Sample Size and Variability

• Larger samples reduce sampling variability (tighten the distribution of sample estimates).
• Visual intuition: as n grows (e.g., 30 → 100 → 1000), the spread of sample proportions around the population value shrinks.
• Variability is highest near 50\% and lower near 0\% or 100\%; mid-range proportions show the greatest uncertainty.

Two-Sample Comparisons (Margin of Error for Differences)

• If comparing two proportions from the same sample (within-group):
  $\text{MOE}_{\text{diff}} \approx 2 \times \frac{1}{\sqrt{n}}$
• If comparing two proportions from independent groups (two separate samples):
  $\text{MOE}<em>{\text{diff}} \approx 1.5 \times \frac{1}{2}\left( \frac{1}{\sqrt{n</em>1}} + \frac{1}{\sqrt{n_2}} \right)$
• Example (same sample): n = 750, split into 375 and 375 ⇒ MOE for each ~ $\frac{1}{\sqrt{375}} \approx 5.2\%$, so $\text{MOE}_{\text{diff}} \approx 2 \times 5.2\% = 10.4\%.$ The reported difference (e.g., 3.6\%) would be far within that margin.

Interpreting Poll Claims

• Steps to evaluate a claim:
  1) Identify claim type: no comparison, within-group, or between two independent groups.
  2) Determine sample size and MOE; compute a 95\% confidence interval for the claim.
  3) Decide whether the claim is supported by the interval (i.e., whether the interval excludes zero for a difference).
• Example framing: a reported lead of 6.8 percentage points from a single poll is not necessarily true if the 95\% CI for the difference includes 0.

Practical Takeaways for Exam/Quiz

• For a single poll percentage (p) with n, assume robustness when p ∈ [30\%, 70\%] and use $\text{MOE} \approx \dfrac{100}{\sqrt{n}}\%.$ For n = 1000, this is about 3.2\%.
• For robust interpretation of minor parties near 0% or 100%, be cautious; margins may be much smaller than the 30-70% rule suggests.
• When reporting, always present the range: $\text{Estimate} \pm \text{MOE}$ to convey uncertainty.
• If splitting the sample into subgroups, recalculate MOE using the appropriate rule of thumb (within same sample vs independent groups).
• Remember: media often omits margin of error; your interpretation should hinge on the CI rather than the point estimate alone.

Quick Reference: Rule of Thumb and range

• Base MOE rule of thumb:
  $\text{MOE} \approx \dfrac{1}{\sqrt{n}} \quad \text{(as a proportion)},$
  or equivalently $\text{MOE (percentage points)} \approx \dfrac{100}{\sqrt{n}}.$
• Valid range: use for p ∈ [30\%, 70\%]; outside this range, the approximation is less reliable.
• For comparing two proportions:
  • same sample: MOE_diff ≈ 2 × (1/√n)
  • independent groups: MOE_diff ≈ 1.5 × average(1/√n1, 1/√n2)
• 95\% confidence means: long-run frequency of intervals capturing the true value is 95\%.