STAT1170 7: Type Errors, Power Analysis + Powers of Statistical Test

Statistical Decision-Making Process

  • Steps include:

    1. Set up a hypothesis.

    2. Decide on significance level ($\alpha$).

    3. Collect data.\sigma

    4. Summarize data.

    5. Use statistical test.

    6. Reject or not reject hypothesis.

    7. Draw conclusions.

Significance Level

  • Represents probability borderline between chance and significance.

  • Common levels: \sigma = 0.05 (5%), \sigma = 0.01 (1%).

Type I and Type II Errors

  • Type I Error (False Positive): Rejecting null hypothesis H0 when it is true; probability is \alpha .

  • Type II Error (False Negative): Not rejecting H0 when it is false; probability is \beta

Example Analysis: Drug Effectiveness

  • Null hypothesis H0: Drug does not work.

  • Type I Error Consequences:

    • Patients misled about drug efficacy.

    • Possible legal actions.

  • Type II Error Consequences:

    • Missed cure, preventable deaths.

Power of a Statistical Test

  • Probability of correctly rejecting H0 when alternative hypothesis H1 is true.

  • Power $= 1 - \beta .

Sampling Distribution

  • Sample means distribution is centered at population mean (\mu) and shrinks with increasing sample size ($n$).

  • Standard error: \frac{\text{sd}}{\sqrt{n}}.

Objectives in Hypothesis Testing

  • Minimize Type I error (\alpha ) often set at 0.05.

  • Maximize Power, aiming 0.80 or minimizing \beta .

Impact of Sample Size on Power

  • Larger samples yield better estimates and increase power.

  • Power increases as mean difference between null and alternative hypotheses increases.

Summary of Findings

  • Power increases with sample size and greater differences between H0 and H1 values.