Study Notes on Statistical Methods: Decision Errors, Error Rates, and Power

Statistical Methods: Decision Errors, Error Rates, and Power

Overview

• Focus on decision errors, error rates, and power in hypothesis testing.

• Learning goals include understanding decision errors and error rates, as well as the concept of power.

Learning Goals

• Decision Errors: Understand the types of statistical errors that can occur in hypothesis testing.

• Error Rates: Comprehend the implications of error rates in research contexts.

• Power: Grasp what power means in statistical hypothesis testing and how it is evaluated.

Decision Errors

• Decision errors arise when making conclusions in hypothesis testing based on sample data.

• Two main types of decision errors:

  • Type I Error (False Positive): Rejecting the null hypothesis (
    $H_0$) when it is true.

  • Type II Error (False Negative): Not rejecting the null hypothesis when it is false.

Decision Errors Table

Definitions of Decision Errors

• Type I Error: Occurs when the null hypothesis is true, yet it is rejected.

• Type II Error: Occurs when the null hypothesis is false, yet it is not rejected.

Analogies to Understand Decision Errors

Court Case Trials

• Hypotheses:

  • $H0$: Defendant is innocent.

  • $HA$: Defendant is guilty.

• Outcomes:

  • Do not convict: Innocent (Correct decision) or Guilty (Type II Error).

  • Convict: Guilty (Correct decision) or Innocent (Type I Error).

Medical Tests

• Hypotheses:

  • $H0$: Patient is healthy.

  • $HA$: Patient is sick.

• Outcomes:

  • Negative Test: Healthy (Correct decision) or Sick (Type I Error).

  • Positive Test: Sick (Correct decision) or Healthy (Type II Error).

Error Rates

Significance Level and Power

• Significance Level ( $ext{Significance Level}$):

  • Defined as the probability of rejecting $H_0$ when it is true, i.e., $P( ext{Type I Error})$.

• Power:

  • Defined as the probability of correctly rejecting $H_0$ when it is false, computed as:
    $Power = P( ext{Reject } H_0 ext{ when } H_0 ext{ is false}) = 1 - P( ext{Type II Error})$.

Balancing Error Rates

• Reducing Type I Error increases Type II Error and vice versa:

  • Key Principle: You cannot simultaneously reduce both error types.

Choosing a Significance Level

• Researchers often select a significance level before conducting tests based on research context and implications of errors.

Example: Manufacturing and Hypothesis Testing

• Machine is intended to fill 1-liter bottles.

• Hypotheses:

  • $H_0$: Machine is working correctly (filling 1.03 liters).

  • $HA$: Machine needs repairs (not filling correctly).

Types of Errors in Manufacturing Example

• Type I Error: Incorrectly concluding the machine is malfunctioning when it is actually operating correctly.

• Type II Error: Failing to conclude the machine is malfunctioning when it indeed needs repair.

Error Rate Calculation

• Management determined that under-filling is a major concern. Significant tests are dependent on the filling volume.

• Given
  $ext{Standard Deviation} = 0.03$ liters, the machine is set to fill at 1.03 liters.

• Sampling every 15 minutes, with a sample of 5 bottles. If the average is less than 1 liter, adjustments are made.

Calculating Type I Error Rate

• The hypotheses for the test are:

  • $H_0: ext{mean } ext{fill} = 1.03$

  • H_A: ext{mean } ext{fill} < 1.03

• Decision Rule: Reject $H_0$ if sample mean < 1 liter.

• Calculate Probability of Type I Error:

  • P(x < 1 ext{ when } ext{mean} = 1.03)

  • Uses normal distribution for computation.

  • Result: Type I Error Rate = $0.0127$

Adjusting Decision Rules

• Inquiry regarding a new decision rule for a 5% significance level.

Sample Size Impact on Type I Error

• Investigation on how increasing sample size (e.g., from 5 to 10 samples) influences Type I Error rates.

Power of a Test Calculation

• Power of a test is defined as:
  $Power = P( ext{Reject } H_0 ext{ when } H_0 ext{ is false})$

• Requires information on actual machine performance to calculate.

Example with Sample Mean of 0.99

• For the test:

  • Hypotheses:

  • $H_0: ext{mean} = 1.03$

  • H_A: ext{mean} < 1.03

• Using actual filling mean of 0.99 liters:

  • Calculate mean and standard error of sampling distribution for ar{x}.

  • Evaluating the sample mean's probability condition when the actual mean is 0.99 liters reveals the power of the test.

Illustrations in Testing

• Graphs may depict Type I Error versus testing power. Labeling and shading the relevant regions assists in visual understanding of statistical relationships and methodologies.