Topic 17a

PSYC 220 - Psychological Statistics

Topic 17a: Analysis of Variance (ANOVA)

Hypothesis Tests

• Types of Hypothesis Tests Based on Number of Samples:

  • One Sample:

  1. Do we know the population standard deviation (σ)?

     • Yes: Z test

     • No: t test (using sample standard deviation)

  • Two Samples:

  • Are the samples independent or dependent?

    • Independent: two independent samples t test

    • Dependent: dependent sample t test

  • Three or More Samples:

  • Are the samples independent or dependent?

    • Independent

    • Dependent: Repeated measures

Motivating Example

• Research Focus:

  • Comparing effects of:

  1. Counseling-only

  2. Drug-only

  3. Counseling-drug combination therapy on depression

• Procedure:

  • Participants assigned to:

  • Only therapy sessions

  • Anti-depressant medication

  • Combination of both treatments

  • Depression levels were measured after 3 months.

• Research Question:

  • “Are there significant differences in depression across the treatment regimens?”

Terminology

• Factor:

  • The independent variable that designates the groups being compared.

• Levels:

  • Individual conditions or values that make up a factor.

• K:

  • Number of levels in the ANOVA.

ANOVA Research Examples

• Example 1:

  • Participants assigned to three groups based on therapy:

  • 1. Counseling-only

  • 2. Drug-only

  • 3. Counseling-drug combination therapy

  • Measured after 3 months for significant differences across groups in depression levels.

• Example 2:

  • Participants received random assignments to three dosage levels of pain-reducing medication:

  • High, moderate, or low dosages.

  • Pain reported on a 10-point scale (0 = no pain, 10 = extreme pain).

  • Research question: “Are there significant differences in reported pain across the groups?”

• Example 3:

  • Subjects randomly assigned to three groups for a reading task:

  • Group 1: No background noise

  • Group 2: Moderate background noise

  • Group 3: Loud background noise

  • Research question: “Are there significant differences in reading comprehension across differing levels of background noise?”

Example Data

• Population:

  • Population 1 (Treatment 1):

  • Population 2 (Treatment 2):

  • Population 3 (Treatment 3):

Hypotheses for ANOVA

• For 3 groups:

  • Null Hypothesis (H0):
    H0: μ1 = μ2 = μ_3

  • Alternative Hypothesis (H1) (if H0 is rejected):

  • Many possible patterns of mean differences, such as:

    • μ1 ≠ μ2 ≠ μ_3 (all 3 means are different)

    • μ1 = μ2 but μ_3 is different

    • μ2 = μ3 but μ_1 is different

    • μ1 = μ3 but μ_2 is different

  • Overall:

    • H1: At least one mean differs.

Why Not Just Run Multiple t-Tests?

• Reason:

  • Performing multiple independent-samples t-tests can lead to an increased probability of committing a Type I error.

• **Example of Error with α Level:

  1. Set alpha to α = .05.

  2. After 3 pairwise comparisons, your experiment-level α would be:**
     α = .05 * 3 = .15

  • For 4 pair-wise comparisons:
    α = .05 * 4 = .20

  • For 9 pair-wise comparisons:
    α = .05 * 9 = .45

Introduction to ANOVA

• Definition:

  • ANOVA (Analysis of Variance):

  • Used to evaluate mean differences between three or more treatments.

• Advantage over t-test:

  • Can compare more than two treatments (groups) at the same time.

Logic of ANOVA

• Key Concept:

  • It is not possible to compute sample mean difference between more than two samples.

  • Instead, ANOVA uses variance to ascertain differences across groups rather than just means.

Test Statistic for ANOVA

• F-test (F ratio):

  • Based on variance, rather than the sample mean difference.

• Variance:

  • A measure of variability (how spread out the numbers are from the mean).

Variance Components

• Between Group Variance:

  • Variability due to treatment (group) differences + variability due to individual differences.

• Within Group Variance:

  • Variability due only to individual differences.

F-Ratio Definition

• Formula:
  F = rac{ ext{Variance (differences) between groups (treatments)}}{ ext{Variance (differences) within groups (treatments)}}

• If H0 is true:

  • Size of the treatment effect is near zero, hence F is near 1.00.

• If H1 is true:

  • Size of the treatment effect is more than 0, hence F is noticeably larger than 1.00.

• Note:

  • The denominator of the F-ratio is called the error term.

  • F cannot be negative.

ANOVA Summary

• Hypotheses:

  • H0: ext{μ}1 = ext{μ}2 = … = ext{μ}_k

  • H1: At least one mean differs.

• Critical Value, Fcrit:

• Test Statistic:

  • Compute F statistic (if H0 is true, then F =1).

• Decision Rule:

  • Reject H0 if F{obs} > F{crit}

• K:

  • Number of groups in the design.

Compute F Statistic

• F-Ratio Formula:
  F = rac{ ext{Variance between groups (treatments)}}{ ext{Variance within groups (treatments)}}

Variance Refresher

• Definition of Variance:

  • Variation is the average squared deviations between an observation and the mean of the data.

• Sum of Squares:

  • Abbreviated as SS.

• Sample Variance General Formula:
  s^2 = rac{ ext{SS}}{n-1}

Compute F-Ratio

• Formulas for F-Ratio:

• F = rac{SS{between}/df{between}}{SS{within}/df{within}}

Degrees of Freedom Analysis

• Within Treatments Degrees of Freedom:

  • df_{within} = N - k

• Between Treatments Degrees of Freedom:

  • df_{between} = k - 1

• Total Degrees of Freedom:

  • df_{total} = N - 1

Summary: F Ratio

• Goal of ANOVA:

• F Ratio Formula:
  F = rac{ ext{Variance between treatments}}{ ext{Variance within treatments}}

• Each variance in the F-ratio is computed as SS/df.

  • SS{between} = rac{SS{between}}{df_{between}}

  • SS{within} = rac{SS{within}}{df_{within}}

• Total variability is analyzed into two components:

  • SS{total} = SS{between} + SS_{within}

  • df{total} = df{between} + df_{within}

ANOVA Summary Table

• Concise presentation of ANOVA results for organization and computation checking.

Critical F

• F Table:

  • Requires both df for between group variance and df for within group variance to find critical F value.

  • Use df for between group variance to identify the column and df for within group variance for the row in the F table.

Hypothesis Testing Steps

1. State the hypotheses.

2. Set the Criteria for Decision (Critical F).

3. Compute Sample Statistics (F).

4. Make a statistical decision and draw a conclusion.

5. Compute and report the effect size.

Example: Cell Phone and Driving

• Research Design:

  • Test the effect of talking on a cell phone on driving behavior.

• Participants:

  • Selected 15 participants; assigned randomly to three groups:

  • 1. “No phone” condition

  • 2. “Hands-free phone” condition

  • 3. “Hand-held phone” condition

• Operationalization:

  • Driving behavior measured as “number of driving mistakes.”

Additional Example Data for ANOVA

Steps for ANOVA

1. State the null and alternative hypotheses.

2. Complete the ANOVA Table.

3. Determine the boundary (critical) value of the rejection region under α level = .05.

4. Make a statistical decision based on observed results.

5. Calculate the effect size.

Post Hoc Tests

• Purpose of Post Hoc Tests:

  • ANOVA compares all individual mean differences simultaneously in one test.

  • A significant F-ratio indicates at least one mean difference is statistically significant but doesn’t specify which means differ.

  • Post hoc tests determine exactly which mean differences are significant.

Measuring Effect Size for ANOVA

• Effect Size Calculation:

  • Compute the percentage of variance accounted for by the treatment conditions.

  • Typically reported as η² (eta squared), indicating the proportion of variance explained by the group (treatment).

Steps Summary for ANOVA

1. State the null and alternative hypothesis.

2. Complete the ANOVA table.

3. Find the critical F value.

4. Compare the observed F with the Critical F, make a statistical decision, and draw a conclusion.