PSYC3010 final exm

What is an interaction

• particular effect or relationship between variables change as a function of another variable

how can interactions extend upon our understanding of established effects/relationships

• add nuance - indicate under what conditions the effect is likely to be stronger, weaker, non-existent, or reversed

advantages of factorial design over one way designs

• allow us to look at effects of two or more factors simultaneously

• allow us to examine individual effects of each factor on DV and interactions between factors

• more economical in terms of participants (e.g. a two-way factorial design has same level of power to detect potential effects of two factors as two one-way experiments)

• assesses generalisability of effect of interest

  • if effect is same across levels of another factor (no interaction) it is generalisable

  • if effect differs across levels of another factor (interaction) it is conditional on the level of that factor)

boundary conditions

conditions under which an effect is more or less likely to emerge

features of factorial designs

• at least two factors

• each factor has at least 2 levels

• participants assigned to at least one level of every factor

• each cell in the design is a different combination until all possible levels of factors and combos are included

• significant interactions and both main effects are independent effects and can occur in any combination

  • the presence of one effect does not indicate the presence or absence of other effects

  • significant interactions may qualify significant main effects

main research questions that can be addressed in two-way ANOVA

• is there a main effect of Factor A on the DV

  • is there a significant difference between the means of the levels of Factor A?

• is there a main effect of Factor B

• Is there a Factor A x Factor B interaction

  • does the effect of Factor A on the DV change at each level of Factor B

  • vice versa

  • would only focus on one

what are simple effects

• the effect of one factor at each level of the other factor

• used if there is a significant interaction

• only use one simple effects (not both) depending on hypothesis

give an example set of simple effects

• simple effects of video game type - does the effect of video game type on anxiety change at each level of breathing method

• simple effects of breathing method - does the effect of breathing method on anxiety change at each level of video game type

what does it mean if a significant interaction qualifies a significant main effect

• suggests effect of factor A changes at each level of factor B

• main effects of Factor A may not be representative of pattern of results

• focus on interpreting interaction and simple effects

how are marginal means calculated

• average of all scores of one factor, regardless of the level of the other factor (collapsing over levels of the other factor/s)

what are marginal means used for

• to detect main effects

• main effects do not tell us where the significant differences are within the factor’s levels

• if there are more than 2 levels of the factor, follow-up tests needed

how are cell means calculated

• average of all scores in one cell (i.e. at one level of Factor A and one level of Factor B)

• separate cell mean for all possible combos of levels

how are 2-way interactions plotted

using cell means

what are cell means used for in a two-way ANOVA

• to detect simple effects

• simple effects test if cell means are significantly different

• only used if we find a significant interaction

• simple effects do not tell us which cell means are significantly different

• if focal factor has more than 2 levels, follow-up tests needed

how are main effects and simple effects different

• main effects - compares marginal means,

  • is there an overall effect of Factor A on the DV

  • is there an overall effect of Factor B on the DV

• simple effects - compares cell means, used to follow up a significant interaction

  • what is the effect of Factor A on the DV at each level of Factor B

  • what is the effect of Factor B on the DV at each level of Factor A

interpret this graph

• disordinal interaction

• playing violent video games increases anxiety only in those engaging in normal breathing, this effect is attenuated when participants engage in deep breathing (simple effects)

• main effect of video game type on anxiety

• unlikely to be main effect of breathing method on anxiety

how do you interpret a significant interaction

with simple effects

interpret this graph

• main effect of lighting on fear

• main effect of ambient noise type on fear

• no interaction

interpret this graph

• main effect of feedback on task performance

• main effect of task difficulty on task performance

• disordinal interaction

• as feedback becomes more negative, task performance worsens. when task difficulty is difficult, this effect is stronger

interpret this graph

• no main effect of presentation format on engagement

• no main effect of age group on engagement

• disordinal interaction

• when presentation format is text-only, teenagers have the lowest engagement. when format is video teenagers have the highest engagement. this effect is reversed for older adults

interpret this graph

• main effect of time of day on positive mood

• no main effect of caffeine intake on positive mood

• ordinal interaction

• simple effects reveal that increasing caffeine improved participants mood when time of day was morning, however this effect reversed when time of day was evening

• simple effects reveal that when time of day was morning participants reported a less positive mood compared to when time of day was evening. However, the magnitude of this effect decreased as caffeine intake increased

notations for factorial designs

• between-participants or between-groups: all factors have different people/independently assigned people in each level

• within-participants or within-groups: all factors have same people across all levels

• Mixed: combination of between-participants and within-participants factors

give an example of a fully between-participants design

3 × 2 ANOVA

Factor A: alcohol consumption (0 drinks, 2 drinks, 4 drinks)

Factor B: distraction (distracted, control)

DV: creativity

Participants randomly allocated to drink either 0, 2, or 4 drinks. After drinking, participants randomly assigned to listen to either no loud music or loud music. All participants then create limericks which are evaluated on creativity by third party

what research question can be addressed with a one-way ANOVA

• is there an effect of the IV on the DV

  • is there a significant difference between mean DV scores across levels of the DV

  • is IV has >2 levels, which means are significantly different

how is variance partitioned in a one-way ANOVA

• between groups variance: systematic variation in the DV scores between the different levels of the IV (treatment variance)

  • i.e. the spread of group means around the grand mean

• within groups variance: random variation in the DV scores that cannot be explained by the IV level (error variance)

  • i.e. the spread of individual DV scores around their respective group mean

what would we see if there was no effect of the IV in a one-way ANOVA

All or most of the variation in DV scores would be error variance

explain the structural model for a one-way ANOVA

X_ij = the score of person i in group j (i.e. individual score)

μ. = grand mean

𝜏_j = treatment effect for group j

ε_ij = error associated with person i in group j (individual’s error)

.: 3 components of a person’s score are the grand mean plus treatment effect for being in a particular group plus error

SS_treatment one-way between-participants ANOVA

• between-participants variability

• sum of squared differences between group means and grand mean multiplied by number of people per group

SS_error in a one-way between-participants ANOVA

• within-groups variability

• sum of squared differences between individual scores and their group mean

MS_treatment in a one-way between-participants ANOVA

• index of variance among group means

• calculated by dividing SS_treatment by df_treatment

MS_error one-way between-participants ANOVA

• index of variance among participants within groups

  • pooled within-cell variance

• calculated by dividing SS_error by df_error

F ratio one-way between-participants ANOVA

F = MS_treatment / MS_error

• measure of treatment variance to error variance

• larger F means more likelihood of treatment effect (must be compared to critical F)

research questions that can be addressed in a two-way ANOVA

• is there a main effect of Factor A on the DV

  • significant difference between mean scores of levels of Factor A?

• is there a main effect of Factor B on the DV

  • significant difference between mean scores of levels of Factor B?

• is there an AxB interaction

  • does effect of Factor A on DV change at each level of Factor B or vice versa

How is variance partitioned in a two-way between participants ANOVA?

• variance due to main effect of Factor A: systematic variance in DV between different Factor A levels

• variance due to main effect of Factor B

• variance due to AxB interaction: systematic variation in DV between different cells that can’t be explained by main effects

• within-groups variance: random variation in DV that can’t be explained by model (error variance)

Two-way between-participants ANOVA structural model

X_ijk = the score of participant i at level j (of Factor A) and level k (of Factor B)

μ. = grand mean

α_j = treatment effect of level j (Factor A)

β_k = treatment effect of level k (Factor B)

αβ_jk = treatment effect of cell jk (interaction effect)

ε_ijk = error

.: participant score is grand mean + treatment and interaction effects relevant to participant + error

SS in two-way between participants ANOVA

SS_A = variability due to main effect of Factor A

SS_B = variability due to main effect of Factor B

SS_AB = variability due to the interaction of the two factors

SS_error = within-groups variability

what is MS

corrected estimate of variance around a mean used to calculate F ratios

MS in two-way between participants ANOVA

• Divide each SS (variability estimate) by respective degrees of freedom

F ratios in two-way between participants ANOVA

• ratio of systematic variance of each effect to unsystemaic variance

• because there are 3 sources of between-groups variance (SS_A, SS_B, SS_AB), 3 F ratios are calculated

calculating F ratio (two-way between participants ANOVA)

treatment variance / error

What is sums of squares (SS)

index of variability around a mean

what does a significant F test tell us (two-way between participants ANOVA)

• significant F tests only tell us there is a significant difference somewhere, not where that difference is

• main effect significant F test - there is a significant difference somewhere among marginal means

  • if the factor has >2 levels, follow-up tests are needed to determine exactly where this difference is

• interaction significant F test - there is a significant difference somewhere among the cell means

  • if either factor has >2 levels, simple effects follow-up tests are needed to determine where

F test for main effect of Factor X (two-way between participants ANOVA)

MS_X / MS_error e.g.

F test for main effect of interaction (two-way between participants ANOVA)

MS_AB / MS_error e.g.

Two-way ANOVA assumptions

• independence of observations - each data point does not influence, relate to, or predict the value of another data point

  • no two measures are drawn from same participant (between-groups only)

• continuous DV

• normality - group populations are normally distributed

• homogeneity of variance - group populations have the same variance

what are omnibus tests

• any test resulting from preliminary partitioning of variance

• looks for any and all possible differences among levels of a factor or combos of factors

what are omnibus tests in a 2-way ANOVA

• main effect of Factor A

• main effect of Factor B

• AxB interaction effect

omnibus tests vs follow-up tests

omnibus tests - tells us if there is a difference anywhere. indicates whether follow-up tests are requires

follow-up tests - tells us where the difference/s are

when would omnibus tests require follow up in a 2-way ANOVA

• significant main effect —> more than two levels —> follow-up required (main comparisons)

• significant interaction —> significant simple effects test —> focal predictor has more than 2 levels —> follow-up required (simple comparisons)

linear vs pairwise comparisons

• linear: compare one mean OR one set of means against another mean or set of means

• pairwise comparisons: compare one mean against another mean

  • all pairwise comparisons are technically linear contrasts

what are main effect comparisons/main comparisons

• follow-up tests for significant main effect

• can use either pairwise or linear comparisons

  • this course focuses on pairwise

how are pairwise comparisons usually conducted

• using t tests

• t-tests use MS_error term from omnibus ANOVA

• t-tests in pairwise comparison focus on the difference between means of two conditions

• t-tests compare means across the levels of the focal predictor/factor (marginal means)

what does a significant t-test for a main comparison mean

• the observed difference between means is significantly larger than what we would expect from chance alone

how are significant two-way interactions followed up in a 2-way ANOVA

• with test of simple effects

what are simple effects

• effects of one factor at each level of the other factor

• compare cell means

• typically only test one set of simple effects based on hypothesis

what does a significant F test for a simple effects test mean

• there is a significant difference somewhere among the cell means of the levels of the focal predictor at the level of the other predictor the F test was significant for

how many F ratios/simple effects does a simple effects follow-up test produce

• always be the same number of simple effects as there are levels of Factor B

when do simple effects need to be followed up

• when there are more than 2 levels of Factor A (the focal predictor for the simple effects)

• only significant simple effects need to be followed up

how are simple effects followed up

• using simple comparisons

how are simple comparisons conducted

using pairwise comparisons

what does a significant t-test for a simple comparison mean and what types of means are being compared

• cell means

• tells us there is a significant difference between one cell mean and another

what is the error term used for all follow-up tests in a factorial between-participants ANOVA

MS_error from the original omnibus ANOVA

why are significance tests not that helpful when assessing importance of findings

• significance tests have binary outcomes

• don’t provide information about practical significance of findings

• tiny, unimportant effects may be deemed significant if large enough sample size, but hold no practical value

what are effect sizes and why are they important

• another way of assessing importance of an effect

• many forms of effect size, each with own rules

what are the 3 main approaches for testing effect sizes of F-tests

• eta squared

• partial eta squared

• omega squared

differences between estimates depends on sample size and error variance

What is eta squared

• proportion of total variance in sample’s DV scores that is accounted for by the effect

• considered biased estimate of true magnitude of actual effect in population (less conservative)

• most commonly reported - easily interpretable

how is eta squared calculated

SS_effect / SS_total

• variability accounted for by effect of interest divided by total variance in DV

what is partial eta squared

• proportion of residual variance in sample’s DV scores

• residual variance - variance left over after removing any variance associated with other factors (i.e. only error variance and variance associated with focal factor)

• most inflated effect size (least conservative)

how is partial eta-squared calculated

SS_effect / (SS_effect + SS_error)

• variability accounted for by effect of interest divided by error variance and variance explained by effect of interest

what is omega squared

• estimated proportion of total variance in DV scores accounted for by effect

• conceptually same as eta-squared, however, includes more parameters to reduce bias (includes df)

• considered less biased estimate of true magnitude of effect

• uses MS as well as SS

how is omega squared calculated

SS_effect - (df_effect x MS_error) / SS_total + MS_error

compare eta-squared and omega-squared

eta-squared
- based on observed variability in sample
- may be biased because only based on one sample

both
- estimating the same thing: how much of the total variance in DV is explained by the effect?

omega-squared
- based on estimated variability within population
- imposes additional constraints leading to slightly smaller estimates to account for bias from using only one sample

compare eta-squared and partial eta-squared

eta-squared
- proportion of total variance explained by effect
- comparable across factors/effects

both
- based on variability in DV (but not same variability)

partial eta-squared
- proportion of residual variance explained by effect after removing variance due to other factors
- not comparable across factors/effects
- tends to be inflated (smaller denominator)

what is Cohen’s d

• most common effect size measure for pairwise comparisons

• measures how many standard deviations apart two means are

when is Cohen’s d used

when calculating effect size for pairwise comparisons (t-tests)

• main effect comparisons

• simple comparisons

how is cohen’s d calculated

difference between means divided by square root of MS_error (standard deviation of MS_error)

what is Confidence Interval

range of values we believe contains ‘true’ value of the thing we’re measuring (i.e. d)

how is CI used to determine significance

• if CI crosses 0, effect is likely non-significant (good chance true value could be 0 i.e. no effect)

• if CI does not cross 0, effect likely to be significant

what is a higher-order design

• factorial designs with more than 2 factors

• allow for designs with higher external validity bc the world is more complicated than one-way or two-way designs

  • more factors = more representative of complex nature of world

• but also increases complexity of patterns of effects and interactions

omnibus effects in a three-way ANOVA

• 3 main effects

• 3 two-way omnibus interactions

• 1 three-way interaction

what research questions can be addressed with a three-way ANOVA

• is there a main effect of Factor A on the DV

  • significant difference between marginal means of levels of Factor A collapsing over Factors B amd C

• is there a main effect of Factor B

• is there a main effect of Factor C

• is there a factor AxB interaction

  • does the simple effect of Factor A on the DV change at each level of Factor B collapsing over Factor C

• is there a factor AxC interaction collapsing over factor B

• is there a Factor AxB interaction collapsing over factor C

• is there a factor BxC interaction collapsing over factor A

• is there a Factor AxBxC interaction

  • does the simple interaction between two factors change at each level of the third factor

how are three-way interactions graphed

plotted as two separate two-way interactions with separate graph for each level of third factor

• y-axis is DV

• x-axis is most important factor

• lines are the second most important factor

• separate graphs are the least important factor

in this three-way interaction, identify the importance of factors

• game type is most important factor (on x-axis)

• breathing method is second most important factor (lines)

• gamer identity is least important factor (separate graphs)

how is variance partitioned in a three-way ANOVA

• variance attributed to main effects (A, B, C)

• variance attributed to omnibus two-way interactions (AxB, AxC, BxC)

• variance attributed to omnibus three way interaction (AxBxC)

• error

explain the components of the structural/linear model of a three way ANOVA

A person’s score is the sum of

• the grand mean

• the main effect of the level of factor A/B/C they are in

• unique two-way interaction effects

• unique effect of being in their particular cell

• error/residual

why can error variance be considered a treatment x participant interaction in a one-way within-participants ANOVA

this takes into account inconsistencies in treatment effect across participants - some participants may be consistently more or less affected by treatment than others

nature/magnitude of treatment effect may change across participants

error terms in a within participant ANOVA

• each omnibus effect uses its own error term

• error term is interaction of effect in question (incl. interactions) and participant factor

compare F calculations in a within-participants ANOVA with a between-participants ANOVA

• error term used in between-participants is pooled variance (MS_error)

• error term used in within participants is individual error term for each effect (MS_AxP, MS_BxP, MS_AxBxP)

assumptions of within-participants ANOVA

• independence of observations - responses from different participants must be independent

• normality of difference scores - difference scores between any pair of treatment conditions should be normally distributed

  • ANOVA robust to this if large sample size

• continuous DV

• sphericity

what is sphericity

for factors with >2 levels, variances of pairwise differences should be roughly equal (=1)

what is Mauchly’s test of sphericity

test to detect violations of assumption of sphericity

if test is significant (p < .005) the sphericity likely violated

but test often fails to detect violations (comes up non-significant)

Mauchly’s test only used for sphericity of individual factors (not interactions)

what are epsilon adjustments

• value by which df for numerator and denominator of F test are multiplied

what are Huynh-Feldt and Greenhouse Geisser adjustments

• calculations to produce the epsilon value by which df will be multiplied

• adjusts critical F not obtained F

what are the differences between the Huynh-Feldt and Greenhouse-Geisser epsilon adjustments

• GG - more conservative (greater adjustment of df)

• HF - less conservative (will result in less adjustment of df)

• HF designed because GG considered too conservative

advantages of fully within-participant designs

• more power: more error variance removed bc we can account for individual differences

  • more likely to find significant effect if it actually exists (reduces Type II error)

• fewer participants needed to detect same sized effect compared to between-participants design

disadvantages of within participant designs

• demand characteristics

• order/sequencing effects

types of order/sequencing effects

• learning/practice effects

• fatigue

• habituation - lower sensitivity in later conditions

• sensitisation - higher sensitivity in later conditions

• contrast - previous conditions set standard for later conditions

how are omnibus F ratios calculated in mixed designs

• 2 different error terms

• Participant variance (MS_participants) used for BP factor main effect

• for WP factor and interaction, the product of participant variance and variance due to interaction is used as error term

  • MS_{WPxparticipant}

compare F ratio terms for within, between, and mixed study designs

• all three use MS of the relevant effect as numerator

• between participants ANOVA uses pooled error variance within all cells (MS_error) for denominator of all F ratios

• within participants ANOVA uses separate error term for each F ratio based on the interaction of the relevant effect with the participant factor

• mixed ANOVA uses participant error term (MS_participant) for denominator of BP main effect, and interaction between WP and participant factor as error term for other two effects (interaction and WP main effect) (MS_{WPxparticipant})

main effect follow-ups (mixed ANOVA)

• main effect comparisons (t-tests) to compare marginal means

• when following up a BP main effect, original error term from omnibus test can be used (MS_participants)

• when following up a WP main effect, separate error term for each comparison must be calculated with only the data involved in the comparisonq