Lecture 3 Notes - Null Hypothesis Tests

Inferential Statistics

• Inferential Statistics: Making inferences about a population from a sample.
  • Example: Sample mean (M) → population mean ($\mu$)
• Confidence Intervals
  • Sample mean M is the best point estimate of $\mu$.
  • Construct a confidence interval around M to estimate the range where $\mu$ likely falls.

Hypothesis Testing

• Alternative method for population inference.
• Starts with a default assumption: null hypothesis (H0).
• Assess if sample evidence is strong enough to reject H0.

Hypothesis Testing (Single Mean, σ Known)

• Statistical Hypotheses:
  • Null Hypothesis (H0): $\mu = 50$ (Mean literacy score is 50).
  • Alternative Hypothesis (H1): $\mu \neq 50$ (Mean literacy score is not 50).
• Outcome: Reject H0 or retain H0.

Null Hypothesis

• Represents a null or conservative state.
• Example: Australian students are no different from American students.
• Refers to a population parameter (e.g., H0: $\mu = \mu0$).
• Examine sample statistic (M) consistency with H0.
• Reject H0 if M is inconsistent with $\mu = \mu0$.

Rejecting or Retaining H0

• Hypothesis testing is conservative.
• Null hypothesis retained unless contrary evidence exists.
• Sampling Distribution: Consider distribution for the mean under H0 (i.e., assuming H0: $\mu = \mu0$ is true).

Level of Significance (Alpha)

• $\alpha$ (alpha): Probability of error experimenter is willing to tolerate.
• Quantifies the likelihood of error; by convention, $\alpha = 0.05$.

Decision Rule

• Reject H0 if the probability of obtaining a sample mean as deviant or more deviant than the one observed, when H0 is true, is less than or equal to $\alpha$.
• Decision Rule: Reject H0 if |z| $\geq$ zc
• Rejecting H0: "Statistically significant" result.

Rejection of H0

• Observed M=58
• If M falls into rejection region: unlikely if $\mu$ were really 50; conclude $\mu>50$.
• Do not conclude $\mu=58$ (even though best point estimate); M=58 is plausible for a range of $\mu$ values, eg 62

Example 1

• n = 36, H0: $\mu =50$, $\sigma = 12$, M = 55
• Step 1 formulate hypotheses
  • H0: $\mu =50$ H1: $\mu \neq 50$
• Step 2 decide on $\alpha$ level, find zC
  • $\alpha$ = .05 zC = 1.96
• Step 3 Find $\sigma$M, and convert M to z
  • $\sigma_M = \frac{\sigma}{\sqrt{n}} = \frac{12}{\sqrt{36}} = 2$
  • $Z = \frac{M - \mu<em>0}{\sigma</em>M} = \frac{55 - 50}{2} = 2.5$
• Step 4 Apply decision rule (two-tailed)
  • Reject H0 if |Z| $\geq$ Zc
  • |2.5| $\geq$ 1.96 so reject H0
• Step 5 Make an appropriate conclusion
  • Evidence suggests, at the .05 level of significance, that Australian 5th graders have literacy levels, on average, higher than US 5th graders.

Example 2

• n = 36, M = 53, $\sigma = 12
• Step 3 Find $\sigma$M, and convert M to z
  • $\sigma_M = \frac{\sigma}{\sqrt{n}} = \frac{12}{\sqrt{36}} = 2$
  • $Z = \frac{M - \mu<em>0}{\sigma</em>M} = \frac{53 - 50}{2} = 1.5$
• Step 4 Apply decision rule: Reject H0 if |Z| $\geq$ ZC
  • |1.5| < 1.96 so retain H0
• Step 5 Make an appropriate conclusion
  • Insufficient evidence to suggest, at the .05 level of significance, that the average literacy level of Australian 5th graders is different from that of US 5th graders.

Comparison between examples

• H0 rejected in the first example because M was further from $\mu0$ and hence provided more evidence against H0 (55-50 = 5 compared with 53-50 = 3)
• $M - \mu0$ as the observed effect size
• it estimates the true effect size $\mu - \mu0$ which represents how wrong the null hypothesis is

Selecting Level of Significance ($\alpha$)

• $\alpha$ = criterion for rejecting or retaining H0
• $\alpha$=.05
  • If H0 is true, 5% of sample means will lead to an incorrect rejection of H0
• $\alpha$=.20
  • If H0 is true, 20% of sample means will lead to an incorrect rejection of H0 - too high an error rate

Type I and Type II Errors

• Rejecting a true H0: Type I error.
• Retaining a false H0: Type II error.

Effect of Alpha on Type I and Type II Errors

• Alpha controls the risk of Type I error.
• Small alpha increases the risk of Type II error.

Additional Considerations

• Balance between Type I and Type II errors.
• Type I errors generally regarded as worse.
• Emphasis on controlling Type I errors: $\alpha$ usually set at .05 or less.

Null Hypothesis vs Experimental Hypothesis

• Null hypothesis is generally not the same as the experimenter’s hypothesis
• Often the experimenter will predict something positive, for example that a new treatment will be effective, so they would be hoping to reject the null hypothesis and get a significant result
• However sometimes the experimenter might predict a null result, so they would be hoping to retain the null hypothesis

Directional Alternative Hypothesis (One-Tailed Test)

• H1: $\mu > \mu0$ or H1: $\mu < \mu0$

When are One-Tailed Tests Used?

• One-tailed tests are appropriate when an effect in the opposite direction to H1 would be of no interest
• effects that are significant with a one-tailed but not with a two- tailed criterion are viewed with suspicion among researchers
• this course will only consider two-tailed tests