(copy) Module 11 - Complex Experimental Designs

Basic Experimental Designs

One independent and one dependent variable

Complex Experimental Designs

• Multiple variables

• Factorial designs: multiple independent variables

• Multivariate designs: multiple dependent variables

Factorial Designs

• Multiple independent variables

• When it’s this design, tells you that the predictors are categorical

  • In a study you have a drug X drug Y and a placebo

    • Those are categories

• The reason we would have a study

  with multiple predictors is almost always because we think that, the effect of one of the predictors on the outcome might depend on what's happening with the other predictor

  • Has main and interaction effects

Multivariate Designs

Multiple dependent variables

2 x 2 Between-subjects Factorial Design

• Two independent variables in the same study

• Different participants in the various conditions

• 4 groups

• (# x #) # refer to the number of levels in each factor

Example of a Factorial Design

How does playing video games influence cognitive functioning?

• IV#1: Game type (Adventure vs. RPG)

• IV#2: Time played (3 hrs vs. 6 hrs)

• Confirmatory hypotheses:

  • The researcher specifies what effects they expect to find

• Exploratory hypotheses:

  • The researcher does not specify what results will be found

You think there's something about the effect of one of them might differ, depending on the other one

Confirmatory/Directional Hypotheses

• The researcher specifies what effects they expect to find

• You have a very specific effect that you think you’re gonna find

Exploratory Hypotheses

• The researcher does not specify what results will be found

• I think, there might be interaction between these 2 predictors, But I'm not sure what it's gonna be

An Example of a Factorial Design: Contingency Table

• Factorial designs have main effects and interaction effects

• This shows cell means and marginal means

Main Effects in Factorial Designs

• Compare all differences between levels of one independent variable across levels of the other

• When we just isolate the effect of one of the predictors

  • Just comparing the rpg game vs the adventure game (ignoring time spent playing)

  • Or just comparing playing for 3hrs vs playing for 6hrs (ignoring the type of game that was played)

Interaction Effects in Factorial Designs

• Combined Effects of independent variables

• An effect of one IV on the DV depending on the level of the other IV

• Describing main effects in reference to each other

• There's a different effect of the game depending on this other variable here

Example: Ice Cream Preference

• Ice cream you prefer on its own

  • = Main Effect

• Preference for what ice cream you want, depending on what you're having (with Pie)

  • ________ ________

Other Variations of Factorial Designs (Specifically 2×3)

• More than two levels for the independent variable

• A main effect does not clarify which levels differ

• Post-hoc tests:

  • Allow researchers to compare the means of the levels of the IV

Post-Hoc Tests

Allow researchers to compare the means of the levels of the IV

Higher-Order Factorial Designs

• Have additional main effects and interactions

• A design with three or more independent variables (2×2×2)

  • 2 (three hours versus six hours)

  • x 2 (adventure versus RPG)

  • x 2 (low versus high delay of gratification) design has six conditions

Repeated Measures Complex Designs

• Participants are exposed to all levels of the independent variable

• Increased statistical power

• Be wary of order effects, interference effects, fatigue, and other confounds

• The same group of participants take part in all of the conditions

Mixed Designs

Includes between-subjects and repeated-measures design features

• Example:

  • We randomly assign participants to study in either a re-reading or retrieval practice condition and have all participants take a test in a hot and cold room.

  • This is a 2 (Between factor: re-reading or retrieval practice study method) x 2 (Within factor: hot and cold room temperature) between-within mixed design

Best Practices in Open Science: How Many Participants Do We Really Need?

• Complex designs require more participants for appropriate power

• Interaction effects require more participants than main effects

• Smaller effects require more participants

• Researchers should focus on conducting larger, well-powered studies