Polling Margin of Error & Sampling: Quick Notes

Key Concepts

Sampling error vs non-sampling error: sampling error arises from observing a sample; non-sampling errors include nonresponse bias, response bias, recall bias, selection bias, etc.
Poll data typically uses categorical data (e.g., Labor vs National) to infer population behavior.
The “margin of error” (MOE) quantifies the sampling error for a reported percentage.

MOE (in percentage points) ≈
\text{MOE} \approx \frac{100}{\sqrt{n}}
for percentages in the 30\% to 70\% range (i.e., when p is not too close to 0 or 100).
Examples:
- n = 400 ⇒ MOE ≈ 100/\sqrt{400} = 5\%.
- n = 1000 ⇒ MOE ≈ 100/\sqrt{1000} ≈ 3.16\% (about 3.2\%).
For minor parties near 0% or 100%, MOE is smaller or the simple rule is less accurate; the exact formula would yield different values.
The MOE reflects only sampling error, not non-sampling errors.

A 95\% confidence interval for a single percentage estimate is:
\text{Estimate} \pm \text{MOE}
Interpretation: If we repeated the poll many times, about 95\% of such intervals would contain the true population value.
The interval accounts for sampling variability; true error also includes non-sampling errors (not captured here).

Larger samples reduce sampling variability (tighten the distribution of sample estimates).
Visual intuition: as n grows (e.g., 30 → 100 → 1000), the spread of sample proportions around the population value shrinks.
Variability is highest near 50\% and lower near 0\% or 100\%; mid-range proportions show the greatest uncertainty.

If comparing two proportions from the same sample (within-group):
\text{MOE}_{\text{diff}} \approx 2 \times \frac{1}{\sqrt{n}}
If comparing two proportions from independent groups (two separate samples):
\text{MOE}{\text{diff}} \approx 1.5 \times \frac{1}{2}\left( \frac{1}{\sqrt{n1}} + \frac{1}{\sqrt{n_2}} \right)
Example (same sample): n = 750, split into 375 and 375 ⇒ MOE for each ~ \frac{1}{\sqrt{375}} \approx 5.2\%, so \text{MOE}_{\text{diff}} \approx 2 \times 5.2\% = 10.4\%. The reported difference (e.g., 3.6\%) would be far within that margin.

Steps to evaluate a claim:
1) Identify claim type: no comparison, within-group, or between two independent groups.
2) Determine sample size and MOE; compute a 95\% confidence interval for the claim.
3) Decide whether the claim is supported by the interval (i.e., whether the interval excludes zero for a difference).
Example framing: a reported lead of 6.8 percentage points from a single poll is not necessarily true if the 95\% CI for the difference includes 0.

For a single poll percentage (p) with n, assume robustness when p ∈ [30\%, 70\%] and use \text{MOE} \approx \dfrac{100}{\sqrt{n}}\%. For n = 1000, this is about 3.2\%.
For robust interpretation of minor parties near 0% or 100%, be cautious; margins may be much smaller than the 30-70% rule suggests.
When reporting, always present the range: \text{Estimate} \pm \text{MOE} to convey uncertainty.
If splitting the sample into subgroups, recalculate MOE using the appropriate rule of thumb (within same sample vs independent groups).
Remember: media often omits margin of error; your interpretation should hinge on the CI rather than the point estimate alone.

Base MOE rule of thumb:
\text{MOE} \approx \dfrac{1}{\sqrt{n}} \quad \text{(as a proportion)},
or equivalently \text{MOE (percentage points)} \approx \dfrac{100}{\sqrt{n}}.
Valid range: use for p ∈ [30\%, 70\%]; outside this range, the approximation is less reliable.
For comparing two proportions:
- same sample: MOE_diff ≈ 2 × (1/√n)
- independent groups: MOE_diff ≈ 1.5 × average(1/√n1, 1/√n2)
95\% confidence means: long-run frequency of intervals capturing the true value is 95\%.