Psych209 LO12

repricable/reproducible

To what extend does a study's result repeat when study is conducted again

replication crisis

out of 100 of famous psych studies, only half were replicated

how the replication crisis happened

• Novel findings more interesting and exciting

• Replicated and null findings more boring, even if still valuable

• Pressure to Publish (Publish or Die), esp for new professors, which may mess with data to get results

file drawer problem

Overestimating a psychological effect by underreporting null findings. A problem for meta-analysis and reviewers if don’t know of null findings

HARKing

(Hypothesizing After the Results are Known), makes it seem you knew all along

P-hacking

Questionable statistical techniques to get p < .05, done on purpose or unintentionally

• Including/excluding participants wouldn’t have normally

• Including/excluding outliers

• Trying other analyses

Inadequate samples

Sample is too small to get an accurate estimate, less replicable

direct replication

Repeat the study exactly as before, or as close to original as possible, only thing that’s different is the sample

conceptual replication

Same general research question, but operationalizations and procedure changes.

Replication-plus-extension

Replicate original study and add new variables for additional research questions

Open science

Sharing your hypothesis, procedures, analyses, and results so that others can check your study

Open data

part of open science, providing all data and analyses (Allows for double-checking)

open materials

providing all manipulations and materials you used in your study (easier for replication)

preregistration

outline of hypothesis, sample recruitment, procedures, and data analysis plans that is posted publicly.

Holds researchers accountable to their original plan (prevent HARKing)