Lecture 7 – Inference for a Population Proportion (Complete Study Notes)

l    Lecture 1 – Sample Proportions

• Overarching course aim: introduce Statistics as the science of collecting, analysing & interpreting data.

• Last lecture recap: Central Limit Theorem (CLT), assumption checking, inference for a mean.

• This week’s roadmap

  • Sample proportions

  • Sampling distribution of counts & proportions

  • Confidence intervals for a proportion

  • Hypothesis tests for a proportion

Why move from means to proportions?

• Real-world variables are often binary (yes/no, success/failure) rather than quantitative & normal.

• CLT still provides an approximate normal distribution for the sample mean of Bernoulli trials when n large.

• Statistical tools always rely on assumptions → must diagnose validity (e.g., normal quantile plot, sample size).

Motivating example – “Mike Promesses” poll

• Needs p=0.5 of votes to win.

• Poll: n=200 respondents, \hat p_{\text{obs}}=0.54.

• Desired outputs:

  1. 95 % Confidence Interval (CI) for true population proportion p.

  2. Hypothesis test H0:p\le 0.5 vs Ha:p>0.5.

• R quick look: prop.test(x = 108, n = 200, p = 0.5, alternative="greater") → \chi^2=1.125,\; p=0.1444, CI [0.479,1].

Binomial model (revision)

• Assumptions (7.1):

  1. Fixed number of trials n.

  2. Two outcomes (success/failure).

  3. Constant success probability p.

  4. Independent trials.

• PMF: P(X=x)=\binom{n}{x}p^{x}(1-p)^{n-x},\;x=0,\dots,n.

• Moments: \muX=np,\;\sigma^2X=np(1-p),\;\sigma_X=\sqrt{np(1-p)}.

Recognising a binomial

• Whenever we “count how many ### in a random sample of fixed size”, the resulting RV is binomial.

• Examples: number of Australians favouring emission cuts, number of female students in SRS of size 4, etc.

Worked exercise – Australian Open Women final

• Assume p=0.2. Sample n=4 → X\sim B(4,0.2).

  • P(X=0)=0.4096 (verified by dbinom).

  • P(X=1)=0.4096, full distribution table presented.

• Moments: \muX=0.8,\;\sigma^2X=0.64,\;\sigma_X=0.8.

Definition 7.2 – Sample proportion

\hat p=\dfrac{X}{n}\quad(\text{a statistic}).

Lecture 2 – Sampling Distribution of Counts & Proportions

Learning outcomes

• [Ch7-L2-O1] Know the sampling distribution of \hat p.

• [Ch7-L2-O2] Understand normal approximation.

Exact distribution of \hat p

• If X\sim B(n,p) then \hat p=X/n takes values {0/n,1/n,\dots,n/n} with same probabilities P(\hat p=x/n)=P(X=x).

• Moments:

\mu{\hat p}=p, \sigma^2{\hat p}=\dfrac{p(1-p)}{n}, \sigma{\hat p}=\sqrt{\dfrac{p(1-p)}{n}}. • Standard error (unknown p): SE{\hat p}=\sqrt{\dfrac{\hat p(1-\hat p)}{n}}.

Example – Lowy emission-reduction poll

• n=1200,\; \hat p_{\text{obs}}=0.63.

• Moments for X: \mu=1200p,\; \sigma=\sqrt{1200p(1-p)}.

• SE: \text{se}=\sqrt{\dfrac{0.63(1-0.63)}{1200}}\approx0.01394.

Normal approximation (Proposition 7.3)

• When n large with np\ge 10 and n(1-p)\ge10, then

  • X\approx N\bigl(np,\,np(1-p)\bigr)

  • \hat p\approx N\Bigl(p,\,\dfrac{p(1-p)}{n}\Bigr).

• Continuity correction improves accuracy for discrete X:

P(X<k)=P\bigl(Y<k-0.5\bigr)\text{ for }Y\sim N(np,np(1-p)).

Example – 50 underground tanks

• Assume p=0.25,\;n=50.

  • \muX=12.5,\;\sigmaX=3.062.

  • P(X<10)=P\bigl(Z<\tfrac{10-12.5}{3.062}\bigr)\approx0.2072.

  • Exact binomial: 0.1637; with continuity correction 0.164 – almost identical.

CLT connection

• Bernoulli variables Bi\in{0,1} with E(Bi)=p,Var(B_i)=p(1-p).

• Sample mean \bar B=\hat p → CLT ⇒ normal approximation.

Rule-of-thumb (Assumptions 7.2)

• Use normal approximation if both np and n(1-p) ≥ 10 (or with \hat p_{\text{obs}} when p unknown).

Data-science interlude – tidyverse

• Introduced pipe %>%, verbs: filter, mutate, arrange, select, relocate, summarise, group_by, join variants.

• Illustrated using palmerpenguins & tiny datasets (band_members, band_instruments).

Lecture 3 – Confidence Intervals for a Proportion

Learning outcomes

• [Ch7-L3-O1] Know CI formula (Wald).

• [Ch7-L3-O2] Check validity assumptions.

• [Ch7-L3-O3] Recognise common CI pattern.

Wald CI (Definition 7.4)

Given random SRS of size n with \hat p{\text{obs}} and C confidence level, CIC(p)=\Bigl[\hat p{\text{obs}}\;\pm\;z^*\sqrt{\dfrac{\hat p{\text{obs}}(1-\hat p_{\text{obs}})}{n}}\Bigr],

where z^ is the C quantile of N(0,1) (e.g., z^=1.96 for 95 %).

Assumptions

  1. Independent observations (SRS or sampling fraction ≤ 1/10 of population).

  2. Approximate normality: n\hat p{\text{obs}}\ge10,\;n(1-\hat p{\text{obs}})\ge10.

Political poll worked example

• n=2951, \hat p=0.51. 95 % CI: [0.492,0.528].

• Interpretation: “We are 95 % confident true Labor support lies between 49.2 % and 52.8 %.”

Additional examples

• Birth-in-spring (108/400) → check np,n(1-p) before CI.

• Australian Open viewing (49/400) → CI \approx [9\%,15.5\%].

Pattern recognition

• All CIs so far: \text{point estimate} \;\pm\; (\text{quantile})\times(\text{SE}).

• Mean with known \sigma: \bar X \pm z^\dfrac{\sigma}{\sqrt n}. • Mean with unknown \sigma: \bar X \pm t^\dfrac{S}{\sqrt n}.

• Proportion: formula above (Wald).

Cautions

• Wald CI can be poor for small n; alternatives: Wilson, Jeffreys, Agresti-Coull (implemented in binomCI).

• CI margin covers random sampling error only – non-response & bias not captured.

• Avoid over-precision; round sensibly (e.g., 54 % \pm 7 %).

Lecture 4 – Hypothesis Tests for a Proportion

Learning outcomes

• [Ch7-L4-O1] Perform Z-test for proportion.

• [Ch7-L4-O2] Check assumptions.

• [Ch7-L4-O3] Link tests \leftrightarrow confidence intervals.

• [Ch7-L4-O4] Recognise Wald test statistic pattern.

Z-test statistic (Definition 7.5)

To test H0:p=p0 vs alternative Ha, use Z=\dfrac{\hat p - p0}{\sqrt{\dfrac{p0(1-p0)}{n}}}.

If assumptions hold ⇒ Z\sim N(0,1) approximately.

Why SE uses p_0 not \hat p

• Under H0 we pretend p=p0 (worst-case). Therefore variance uses p_0.

Assumptions

  1. Independence (SRS & population ≥10×sample).

  2. Approx normality with null value: np0\ge10,\;n(1-p0)\ge10.

Worked examples

  1. Births in spring (108/400): - p0=0.25, one-sided Ha:p>0.25.

    • z_{\text{obs}}=0.923,\; p=0.178 → fail to reject; no evidence.

  2. Voting intention change 2013 vs 2016: - n=2951, p_0=0.465, \hat p=0.51, two-sided.

    • z_{\text{obs}}=4.901, p\approx9.5\times10^{-7} → very strong evidence of change.

  3. TV ratings (49/400) vs p0=0.0688: - z{\text{obs}}=4.243, p=2.2\times10^{-5}.

    • 95 % CI [0.090,0.155] excludes 0.0688 ⇒ same conclusion – MATH1041 students differ.

Relationship test \leftrightarrow CI (Proposition 7.4)

• For two-sided tests at level \alpha, rejecting H_0 ⇔ null value outside 100(1-\alpha)\% CI.

• One-sided tests correspond to one-sided CIs.

Wald statistic pattern

\text{Test statistic}=\dfrac{\text{estimator}-\text{hypothesised value}}{SE}.

Applies to Z-tests (mean or proportion) & t-tests (when \sigma unknown).

Continuity Correction Summary

• Used when approximating discrete binomial by continuous normal.

• Rule: replace integer cutoff k with k\pm0.5 depending on inequality.

• Greatly improves tail probability accuracy (demonstrated with tank example).

Keywords & Take-away Concepts

• Proportion, sample proportion \hat p.

• Binomial distribution & parameters (n,p).

• Sampling distribution & CLT.

• Normal approximation criteria np, n(1-p)\ge10.

• Standard error SE_{\hat p}.

• Wald confidence interval.

• Z-test for a single proportion.

• Continuity correction.

• Tidyverse data-wrangling verbs (filter, mutate, …).

Practical R Commands Cheat-Sheet

• Compute exact binomial prob.: dbinom(x,n,p); cumulative pbinom(k,n,p).

• Normal approx: pnorm(value, mean, sd) or qnorm(prob) for quantiles.

• prop.test(x,n,p0, alternative="two.sided", conf.level=0.95, correct=TRUE) – gives CI & test (with continua­­ty-corr \chi^2).

• binom.test(x,n,p=p0) – exact binomial test & exact CI.

• Standard CI by hand:

se <- sqrt(p_obs*(1-p_obs)/n)
  ci <- p_obs + c(-1,1)*z_star*se

Common Pitfalls & Best Practices

• Do NOT apply normal approximation when np or n(1-p) < 10. Use exact binomial methods instead.

• Interpret CIs in context; avoid overstating precision.

• Report assumptions explicitly (independence, sampling design, sample size adequacy).

• Remember CI ≠ probability interval for parameter; 95 % refers to long-run coverage.

• Wald CI unreliable for extreme \hat p or small n → prefer Wilson/Agresti-Coull.

Connections to Further Topics

• Two-sample comparisons of proportions will extend these ideas (not covered here).

• \chi^2 tests of independence generalise to multi-category tables.

• Generalised linear models (logistic regression) model proportions with covariates.

Self-Reflection Prompts (Exercise 7.2)

• Which part of the inference workflow (formulating H0/Ha, checking assumptions, computing SE) still feels unclear?

• Identify a binary variable in your discipline and outline how you would design a study, compute \hat p, draw a CI, and test a claim.