lecture 3: Hypothesis Tests (Proportions)

Hypothesis Tests Overview

• Framework to evaluate hypotheses using data.
• Null Hypothesis ($H_0$): A simple default, assumed true until contradicted. Cannot be accepted or proven.
• Alternative Hypothesis ($H<em>1$ or $H</em>A$): States a difference or effect exists.
• Logic: Assume $H<em>0$ is true. If data significantly contradicts $H</em>0$, reject $H<em>0$. Otherwise, fail to reject $H</em>0$.
• Four Parts: Null (and Alternative) Hypothesis, Test Statistic, P-Value, Conclusion.
• Test Statistic ($Z$): Measures how many standard errors the sample proportion is from the hypothesized population proportion.
• P-Value: Probability of observing data as extreme or more extreme than the sample data, assuming $H<em>0$ is true. A smaller P-value provides stronger evidence against $H</em>0$.
• Conclusion: Reject $H<em>0$ if $|Z| ext{ extgreater } Z^*$ (e.g., $1.96$ for $95\%$ confidence) OR P-value $ext{ extless }= ext{ }\alpha$ (significance level, e.g., $0.05$). Fail to reject $H</em>0$ otherwise.
• Statistical Significance: When $H_0$ is rejected, the results are statistically significant (likely a real difference, not just sampling noise).
• Inconclusive: When $H_0$ is not rejected, the difference is indistinguishable from sampling noise.
• Direction of Difference: If $H<em>0$ is rejected, the direction (e.g., greater or smaller) is indicated by the sample measurement's sign relative to the hypothesized value (e.g., $ext{\hat{p} - p</em>0}$ or $ext{\hat{p}<em>1 - \hat{p}</em>2}$).

One-Sample Z-Test for Proportions

• Purpose: Test if a population proportion ($p$) is equal to a hypothesized value ($p_0$).
• Null Hypothesis ($H<em>0$): $p = p</em>0$
• Alternative Hypothesis ($H<em>A$): $p \neq p</em>0$
• Test Statistic (Z-score): $Z = \frac{\hat{p} - p<em>0}{\sqrt{\frac{p</em>0(1-p_0)}{n}}}$ where $\hat{p}$ is the sample proportion and $n$ is the sample size.
• P-Value Calculation: For a two-tailed test, $2 \times ext{NORM.S.DIST(-ABS(Z), TRUE)}$.

Two-Sample Z-Test for Proportions

• Purpose: Test if two population proportions ($p<em>1$, $p</em>2$) are equal.
• Null Hypothesis ($H<em>0$): $p</em>1 = p<em>2$ (or $p</em>1 - p_2 = 0$)
• Alternative Hypothesis ($H<em>A$): $p</em>1 \neq p<em>2$ (or $p</em>1 - p_2 \neq 0$)
• Test Statistic (Z-score): $Z = \frac{(\hat{p<em>1} - \hat{p</em>2}) - 0}{\sqrt{\hat{p}<em>{pooled}(1-\hat{p}</em>{pooled})(\frac{1}{n<em>1} + \frac{1}{n</em>2})}}$ where $\hat{p}<em>{pooled} = \frac{x</em>1 + x<em>2}{n</em>1 + n_2}$ is the pooled sample proportion.

Sampling Assumptions and Real-World Challenges

• Key Assumptions: Randomization, independence (between items AND samples), appropriate sample size.
• Real-World Challenges: These assumptions are often violated.
  • Sampling Bias: Non-random or unrepresentative samples lead to inaccurate conclusions (e.g., Big Data Paradox).
  • Self-Reporting Bias: Subjects may provide dishonest or inaccurate information.
  • Interference: Subjects can influence each other, violating independence.
• Mitigation: Design studies carefully to limit bias; be critical of results from poorly designed experiments.