PSYC 301: Module 8

What major statistical methods were covered in PSYC 301?

• Independent samples t-test

• Repeated measures t-test

• One-way ANOVA (between & repeated measures)

• Two-way ANOVA (between-subjects)

• Correlation

• Simple linear regression

What types of designs were omitted from PSYC 301?

• Two-way repeated measures ANOVA

• Two-way mixed-design ANOVA

• Factorial ANOVAs with 3+ IVs

• Multiple linear regression

Why are three-way ANOVAs more complex than two-way ANOVAs?

They involve 3 main effects, 3 two-way interactions, and 1 three-way interaction

What does a three-way interaction mean conceptually?

A two-way interaction between two IVs differs across levels of a third IV

What is multiple linear regression?

A regression model with two or more continuous predictors, each with its own slope, used to predict a continuous outcome

What are the two central analytical approaches emphasized in PSYC 301?

ANOVA and Regression

What additional analytical possibilities exist beyond ANOVA and regression?

Analyses for other data types and research questions (e.g., categorical outcomes, longitudinal data, multilevel data)

What are Abelson’s MAGIC criteria?

A framework describing factors that determine the persuasive impact of a research claim

What does MAGIC stand for?

• Magnitude

• Articulation

• Generality

• Interestingness

• Credibility

Why is magnitude difficult to interpret?

• Large effects are not always practically meaningful

• Small effects can still be theoretically or practically important

When can small effects be impressive?

When they have important conceptual implications or accumulate meaningful real-world consequences

What is articulation in the context of MAGIC?

The clarity, efficiency, and accuracy with which statistical results are translated into meaningful conclusions

Why does articulation become harder with complex designs?

More complex designs allow multiple valid interpretations, requiring careful analytical choices

What is generality?

The extent to which a research finding applies beyond a single study

What are four forms of generality?

• Across studies (replication)

• Across researchers

• Across populations and contexts

• Across operationalizations of IVs and DVs

What statistical tool is used to assess generality across studies?

Meta-analysis

What makes a research finding interesting?

Its ability to change beliefs, resolve disputes, or generate new understanding

How can surprise contribute to interestingness?

Through unexpected magnitude or novelty of effects

How does importance contribute to interestingness?

Via practical implications or broad theoretical relevance

What is credibility in research?

The extent to which a claim is believable given theory, method, and evidence

What contributes to conceptual credibility?

• Logical coherence

• Fit with existing theory

• Fit with existing evidence

• Consistency with common sense

What threatens methodological credibility?

• Data fishiness

• Inappropriate statistical procedures

• Alternative explanations

• Poor construct measurement

What validity framework is referenced at the end of Module 8?

Cook and Campbell’s (1979) validity typology

What is statistical conclusion validity?

Whether the IV is statistically related to the DV

What is internal validity?

Whether the IV–DV relationship is causal

What is construct validity?

Whether the IV and DV accurately represent their intended constructs

What is external validity?

Whether the IV–DV relationship generalizes across contexts and populations

What is the key takeaway of Module 8?

Statistical analysis is necessary but not sufficient—strong research claims require sound statistics, theory, method, and interpretation working together