lecture 8 - comparing 2 means

ANOVA Overview

  • ANOVA = Analysis of Variance

  • Tests for mean differences across two or more groups or treatments

  • Use when comparing 2+ means, with one categorical independent variable (IV) and one continuous dependent variable (DV)

  • Goal: Determine if there's a significant mean difference across groups

Types of ANOVAs

  • Between-subjects ANOVA: Compares means from different groups.

  • Within-subjects ANOVA: Compares means within the same group of subjects.

  • One-way ANOVA: Examining one IV (e.g., mean scores of vocabulary across different age groups).

  • Factorial ANOVA: Examining two or more factors (e.g., vocabulary across age and gender).

Focus for Today

  • One-way Between-subjects ANOVA: Example comparing happiness levels across three movie conditions:

    • After a scary movie

    • After a romantic comedy

    • After a cartoon movie

Key Terminology

  • Dependent Variable (DV): Happiness levels

  • Independent Variable (IV): Type of movie watched

  • Factor: The IV, with levels indicating the different conditions (3 levels: scary, romantic, cartoon).

Statistical Hypotheses

  • Null Hypothesis (H0): m1 = m2 = m3 (means are equal).

  • Alternative Hypothesis (H1): At least one mean is different.

Experiment-wise Alpha Level

  • Concept of experiment-wise alpha level or family-wise alpha level.

  • With multiple tests, cumulative Type I error risk increases.

  • Independent tests reduce no Type I errors to about 85.7% after three tests, increasing Error risk to 14.3%.

  • ANOVA addresses this by comparing all means in one test, maintaining Type I error at 5%.

Logic of ANOVA

  • Calculate Total Variance and separate it into:

    • Between-treatments variance

    • Within-treatments variance

ANOVA F Statistic

  • The F statistic helps show if treatment has a significant effect.

  • Calculated using variance:

    • F = Variance between sample means / Variance within samples

    • High F value indicates significant differences among group means.

One-Way ANOVA Example

  • Consider three samples from different movie conditions.

  • Calculate mean values per condition and assess between-treatment and within-treatment variance.

Assumptions of ANOVA

  • Independence of observations

  • Normal distribution of the residuals

  • Homogeneity of Variance (Levene's test)

    • If violated, consider alternative tests.

Output and Interpretation

  • Report on ANOVA results including F-statistic, p-value, and effect size (e.g., Eta squared (η²)).

    • Example: F(2,33) = 2.51, p = .097, η² = 0.13

  • Eta squared interpretation:

    • 0.01 = Small effect

    • 0.06 = Medium effect

    • 0.14 = Large effect

Post Hoc Tests

  • Necessary when the main ANOVA is significant to determine specific group mean differences.

  • Conduct pairwise comparisons (e.g., scary vs romantic, scary vs cartoon).

  • Use Bonferroni correction to adjust alpha level for multiple comparisons (threshold alpha = .016).

Bonferroni Correction

  • Adjust significance to control Type I error; e.g., p < .05 / # of comparisons = adjusted alpha.

  • Common approach for maintaining Type I error under control.

Final Notes

  • Understanding ANOVA applications crucial for statistical analysis in research.

  • Skills include running, interpreting, and reporting ANOVA and regression analysis, effect sizes, and post-hoc tests.


ANOVA Overview

What is ANOVA?

  • ANOVA stands for Analysis of Variance and is a statistical method used to determine if there are any statistically significant differences between the means of three or more independent (unrelated) groups.

  • This analysis is particularly useful when comparing means from multiple groups, where a single categorical independent variable (IV) is being examined against a continuous dependent variable (DV).

  • Goal: The primary objective of ANOVA is to assess whether the variation among sample means is larger than would be expected by chance alone; this is typically expressed as determining if there is a significant mean difference across groups.

Types of ANOVAs

  • Between-subjects ANOVA: This variant compares means from different groups—each group is composed of different subjects. For instance, participants may be divided based on different experimental treatments.

  • Within-subjects ANOVA: Also known as repeated measures ANOVA, it compares means within the same subjects across different conditions or times. For example, a study might measure the same group of subjects before and after an intervention.

  • One-way ANOVA: Involves examining one independent variable (IV). An example could be examining mean scores of vocabulary tests across various age groups.

  • Factorial ANOVA: In this method, two or more independent variables are examined simultaneously—such as studying vocabulary scores across different age groups and gender.

Focus for Today

  • The focus today is on One-way Between-subjects ANOVA illustrated through an example that compares happiness levels across three different movie conditions:

    • After viewing a scary movie

    • After watching a romantic comedy

    • After enjoying a cartoon movie

Key Terminology

  • Dependent Variable (DV): The variable that is tested and measured in an experiment; in this case, it's the happiness levels of the participants.

  • Independent Variable (IV): The variable that is manipulated to observe its effect on the dependent variable; here, it's the type of movie watched.

  • Factor: Represents the independent variable, where levels of this factor indicate the different conditions present in the study (3 levels in this example: scary, romantic, and cartoon).

Statistical Hypotheses

  • Null Hypothesis (H0): The hypothesis stating that there are no differences among means, i.e., m1 = m2 = m3 (means are equal).

  • Alternative Hypothesis (H1): The hypothesis indicating that at least one mean is different from the others, suggesting that the type of movie affects happiness levels.

Experiment-wise Alpha Level

  • The concept of experiment-wise alpha level, or family-wise alpha level, is significant in multiple comparisons. When conducting multiple tests, the cumulative risk of committing a Type I error (falsely rejecting the null hypothesis) increases.

  • For instance, if we conduct multiple independent tests, the rejection rate can be reduced to approximately 85.7% after three tests, which increases the risk of error to about 14.3%. ANOVA tackles this issue by allowing a single test that compares all means, maintaining the Type I error rate at a predefined significance level (usually 5%).

Logic of ANOVA

  • ANOVA operates on the principle of partitioning total variance into:

    • Between-treatments variance: Variance attributed to the differences among group means.

    • Within-treatments variance: Variance attributed to differences within each group, indicating individual subject variability.

ANOVA F Statistic

  • The F statistic serves as a key indicator in ANOVA to demonstrate whether treatment effects are significant. It is calculated as:

    • F = Variance between sample means / Variance within samples

  • A high F value suggests significant differences among group means, implying that at least one group differs significantly from the others.

One-Way ANOVA Example

  • Consider a study involving three samples from the different movie conditions mentioned earlier.

  • The means for each condition would be calculated while assessing both between-treatment and within-treatment variances to draw conclusions about differences in happiness levels.

Assumptions of ANOVA

  • For the results of ANOVA to be valid, certain assumptions must be met:

    • Independence of observations: The data collected should be independent of each other.

    • Normal distribution of the residuals: The residuals (differences between observed and predicted values) must be approximately normally distributed.

    • Homogeneity of Variance: Assumed that variances in each group being compared are equal; this can be checked using Levene's test. If these assumptions are violated, it might be necessary to consider alternative statistical tests.

Output and Interpretation

  • Reporting ANOVA results includes several key components such as the F-statistic, associated p-value, and effect size (e.g., Eta squared (η²)). An example report might state: F(2,33) = 2.51, p = .097, η² = 0.13.

  • Eta squared interpretation:

    • 0.01 = Small effect

    • 0.06 = Medium effect

    • 0.14 = Large effect

Post Hoc Tests

  • If the main ANOVA indicates significant effects, post hoc tests become necessary to pinpoint which specific group means differ from one another. Common methods include conducting pairwise comparisons, such as comparing scared vs. romantic and scared vs. cartoon movies.

  • Bonferroni correction: This adjustment method is often applied to control for Type I error in multiple comparisons. The significance level (alpha) is divided by the number of comparisons conducted (e.g., adjusted alpha = p < .05 / number of comparisons).

Final Notes

  • Understanding ANOVA applications is crucial for statistical analysis in various research fields. Skills essential in this context include running, interpreting, and reporting ANOVA procedures, insight into regression analysis, calculating effect sizes, and performing post-hoc tests to substantiate findings.