Notes on Hypothesis Testing and Confidence Intervals (Proportions)

Hypothesis Testing, Normal Approximation, and Confidence Intervals (Proportions)

• Test statistic and decision context

  • Our test statistic is based on the sample and is used to decide about the null hypothesis. It should reflect the direction of the alternative hypothesis. If we reject the null, the conclusion is not simply "we rejected H0" but rather a population-level statement like "there is evidence that the parameter lies in the alternative direction".
  • The decision (reject or fail to reject) is the outcome of applying a pre-specified rule (significance level) to the observed statistic; the conclusion should be about the population parameter, not just the decision.
  • When working through a problem, choose a consistent path for the test statistic and stay with it; switching paths mid-analysis (e.g., from counts to proportions to different normalizations) can lead to inconsistent p-values.

• The standard normal distribution (Z)

  • The standard normal distribution is identified by mean 0 and standard deviation 1.
  • Cumulative distribution function: F_Z(z) = P(Z \le z) where $Z \sim N(0,1)$.
  • Percentiles: values of $z$ such that $F_Z(z) = \alpha$ for a given tail probability $\alpha$.
  • Example tail probabilities:
  • $P(Z \le -3) \approx 0.0013$.
  • A common range: about $P(-2 \le Z \le 2) \approx 0.954$.

• Reading and using the standard normal table

  • To find the probability that a standard normal is between two points, use the table for the cumulative probability up to the left endpoint and up to the right endpoint, then take the difference if needed.
  • If you are looking for the probability to the left of a point, you read the cumulative value directly from the table.
  • When using a left limit like $-\infty$, the cumulative probability approaches 0.
  • If you have a problem with a segment like from $-\infty$ to a finite z, you use the corresponding $F_Z(z)$ value.

• Understanding and computing percentiles

  • The cumulative percentage for a value on the left of the distribution is the same as the corresponding percentile of the standard normal.
  • The 75th percentile corresponds to $z \approx 0.674$.
  • For a 95% CI, the critical z-values are $z{0.025} \approx -1.96$ and $z{0.975} \approx 1.96$.
  • Excel or similar tools can provide these critical values easily (e.g., $-1.96$ and $+1.96$ for 95% confidence).

• From counts to proportions and the normal approximation

  • When data are counts (e.g., number of events), under the null we often approximate the distribution by a normal with mean $\mu = np$ and variance $np(1-p)$, possibly with a continuity correction.
  • Continuity correction example: to approximate $P(X \le k)$ for a binomial $X \sim \text{Bin}(n,p)$, use P(X \le k) \approx P\left(Z \le \frac{k + 0.5 - np}{\sqrt{np(1-p)}}\right) where $Z \sim N(0,1)$.
  • When focusing on a proportion, the natural statistic is the sample proportion $\hat{p} = X/n$ and its distribution under the null can be approximated by a normal with mean $p$ and variance $p(1-p)/n$.

• Test statistics for a proportion

  • One common one-sided or two-sided z-test statistic for a null proportion $p0$ is: z = \frac{\hat{p} - p0}{\sqrt{\dfrac{p0(1-p0)}{n}}}
  • If you instead model using the binomial, the exact distribution is $X \sim \text{Bin}(n,p_0)$ with $\hat{p} = X/n$, and you can compute a p-value from that exact distribution or use the normal approximation above.
  • If you choose to work with counts, you can report a p-value based on the binomial model; if you choose to work with proportions, you’ll typically use the normal approximation for the p-value.
  • The literature often emphasizes: once you choose a test statistic (count vs. proportion, normal vs. binomial), stay with that choice through the analysis.

• An example framework: testing a proportion in practice

  • Hypotheses: e.g., $H0: p = p0$ vs. $HA: p < p0$ (one-sided) or $HA: p \neq p0$ (two-sided).
  • Under $H_0$, the sampling distribution of the test statistic is determined (binomial exact or normal approximation).
  • You can compute the p-value either exactly (binomial) or approximately (normal) depending on the setting and sample size.
  • Continuity correction may be used for the binomial-to-normal approximation to improve accuracy.
  • Example numbers from a problem: a binomial count with mean $\mu = 186.75$ and standard deviation $\sigma = 11.83$ for some n and p under $H_0$; in a different setup, you might have $\hat{p} = \frac{60}{747.08}$ as an observed proportion.
  • When using the normal approximation, you compute $z$ as in the formula above and then obtain a p-value from $\Pr(Z \le z)$ (or two-sided $2 \min{\Pr(Z \le z), \Pr(Z \ge z)}$).
  • A continuity correction example: for a binomial count around 60, use 60.5 when converting to a normal for the p-value.

• Proportion testing: choosing between binomial and normal approaches

  • If you can rely on a binomial model (discrete counts), you may compute exact p-values from the binomial distribution.
  • If sample size is large enough and the normal approximation is appropriate, you can use the z-based approach with either $p_0$ or $\hat{p}$ in the standard error, depending on convention.
  • The two approaches should give close p-values when the normal approximation is appropriate; the exact binomial p-value is typically preferred for small samples or when the normal approximation would be poor.
  • The choice of test statistic (count vs. proportion) matters for interpretation, because the parameter of interest is either a proportion ($p$) or a count; using $\hat{p}$ keeps the statistic on the same scale as the parameter.

• Confidence intervals for a population proportion

  • Confidence level and critical values:
  • Choose a confidence coefficient (e.g., 0.95).
  • The corresponding critical z-values are $z{\alpha/2}$ and $z{1-\alpha/2}$; for 95% CI, $z{0.025} = -1.96$ and $z{0.975} = 1.96$.
  • A common normal-approximation CI for a proportion is:
    ext{CI}{0.95} = \hat{p} \pm z{0.975} \sqrt{ \dfrac{\hat{p} (1 - \hat{p})}{n} }
  • If desired, one can formulate CI using the null-proportion standard error or use more exact methods (e.g., Wilson score interval, Clopper–Pearson) depending on the problem context.
  • The CI is constructed for a parameter of interest (here, the population proportion $p$) and provides a range of plausible values for that parameter given the data.

• Practical workflow and interpretation

  • Step 1: Form hypotheses about the parameter of interest (often a proportion $p$).
  • Step 2: Choose a test statistic (count vs. proportion) and the corresponding distribution under the null (binomial exact or normal approximation).
  • Step 3: Compute a p-value (exact or approximate) and compare to the significance level to make a decision.
  • Step 4: If not rejecting, report the p-value and consider a confidence interval for estimation.
  • Step 5: If rejecting, phrase conclusions in terms of the population parameter (e.g., there is evidence that $p$ is less than/greater than/ differs from a specified value).
  • Step 6: When estimating, construct a confidence interval for $p$ to quantify uncertainty.

• Additional notes on interpretation and practice

  • The normal table and z-values are tools to connect observed data to probabilities and decisions.
  • Inferences about a proportion rely on the distributional assumptions (binomial or normal approximation); verify that assumptions are reasonable for the problem at hand.
  • Real-world relevance: hypothesis tests help assess whether a measured proportion of events (e.g., failures, deaths, disease prevalence) is consistent with a hypothesized rate, while confidence intervals quantify the precision of the estimated rate and the range of plausible values for policy or scientific interpretation.

• Quick references to formulas (LaTeX)

  • Z-statistic for a proportion under $H0$: z = \frac{\hat{p} - p0}{\sqrt{\dfrac{p0(1-p0)}{n}}}
  • Standard error with sample proportion: SE_{\hat{p}} = \sqrt{ \dfrac{\hat{p}(1-\hat{p})}{n} }
  • 95% confidence interval for a proportion (normal approximation): \hat{p} \pm z_{0.975} \sqrt{ \dfrac{\hat{p}(1-\hat{p})}{n} }
  • Continuity-corrected binomial-to-normal approximation: P(X \le k) \approx P\left(Z \le \frac{k + 0.5 - np}{\sqrt{np(1-p)}}\right)

• Quick recap of key ideas

  • Hypothesis testing uses a test statistic to decide about $H_0$ with a specified error rate; conclusions should reflect population parameters, not just the decision.
  • The standard normal distribution underpins many tests and confidence intervals; knowing how to read and apply the table (including percentiles like 1.96 for 95% CI) is essential.
  • For proportions, you can work with counts (binomial) or proportions (normal approximation); keep the path consistent and interpret results in terms of $p$.
  • Confidence intervals provide a direct link from testing to estimation, offering a range of plausible values for the target parameter.

• Notes on terminology and practice from the transcript

  • The discussion highlights that the first step in many problems involves formulating hypotheses about a proportion and understanding the distribution of the test statistic under the null.
  • It also emphasizes the practical use of standard normal tables and their associated critical values to determine p-values and confidence limits.
  • The idea of choosing a test statistic (count vs. proportion) and sticking with it is underscored to avoid inconsistency in p-value calculations.