Understanding Alpha Value, Hypothesis Testing, and Errors

Determining Statistical Significance: Alpha Value and Error Types

Alpha Value: Risk Assessment

  • Alpha value represents the level of risk we're willing to take of drawing the wrong conclusion when determining if there is a significant difference between two groups.
  • It addresses how bad it would be to incorrectly conclude a significant difference exists (e.g., based on the amount of drink or milk pregnant mothers consume).

Null Hypothesis

  • Null hypothesis (H₀) states that there is no difference between two groups; any observed difference is due to chance.
  • Expressed as: \mu1 = \mu2 (sample mean 1 equals sample mean 2).
  • Hypothesis testing aims to reject the null hypothesis by gathering evidence that differences between groups are real and caused by the independent variable.

Type I and Type II Errors

  • Type I Error ($\alpha$):

    • Rejecting the null hypothesis when it is actually true.
    • Alpha level represents the probability of making a Type I error.

  • Type II Error ($\beta$):

    • Retaining the null hypothesis when it is actually false (i.e., failing to detect a real difference).
DecisionNull Hypothesis is TrueNull Hypothesis is False
Reject Null HypothesisType I ErrorCorrect Decision
Retain Null HypothesisCorrect DecisionType II Error

Egregiousness of Type I Error

  • Type I error is generally considered more serious.
  • Example: Claiming a cancer treatment is effective when it is not (false positive).

Analogy for Type I and Type II Errors

  • Type I error: Telling a non-pregnant man he is pregnant (false positive).
  • Type II error: Telling a pregnant woman she is not pregnant (false negative).

Accuracy and Conservative Alpha Levels

  • Goal: Be accurate in decisions regarding the null hypothesis.
  • Conservative alpha levels are preferred to minimize Type I errors.

Common Alpha Levels

  • Social Science Research: Commonly set alpha at 0.05 (5% chance of Type I error).
    • 0.05 = 5\%
  • Medical Research: Uses more conservative alpha levels (e.g., 0.01 or 0.001) because the consequences of error are more severe.

Impact of Alpha Level on Significance

  • More conservative alpha levels (e.g., 0.001) reduce the chance of Type I error but make it harder to find statistically significant differences.
  • A large effect between groups is needed to achieve significance with a conservative alpha level.

Examples of Alpha Level Application

  • Social interventions (e.g., in schools) often use \alpha = 0.05 because the consequences of a Type I error are less severe.
  • Drug research and cancer treatments require lower alpha levels (e.g., 0.01 or 0.001) due to the critical nature of the decisions.

T-Distribution Critical Values Table

  • T-distribution tables are used to determine statistical significance.
  • To find the T critical value, the alpha level and whether it's a one-tailed or two-tailed hypothesis need to be known.
  • Common alpha values in the table: 0.05, 0.025.