Chp. 14 Verms - Replication, Credibility, and Wider Importance

replicable

(also reproducible) describing a study whose results have been reproduced when the study was repeated, or replicated.

direct replication

a replication study in which researchers repeat the original study as closely as possible to see whether the original effect shows up in the newly collected data

conceptual replication

a replication study in which researchers examine the same research question (the same conceptual variables) but use different procedures for operationalizing the variables

replication-plus-extension

a replication study in which researchers replicate their original study but add variables or conditions that test additional questions

scientific literature

(also literature) a series of related studies, conducted by various researchers, that have tested similar variable

meta-analysis

a way of mathematically averaging the effect sizes of all the studies that have tested the same variables to see what conclusion that whole body of evidence supports

file drawer problem

a problem relating to literature reviews and meta-analyses based only on published literature, which might overestimate the support for a theory because studies finding null effects are less likely to be published than studies finding significant results and are thus less likely to be included in such reviews

HARKing (hypothesizing after the results are known)

a questionable research practice in which researchers create an after-the-fact hypothesis about an unexpected research result, making it appear as if they predicted it all along

p-hacking

a family of questionable data analysis techniques, such as adding participants after the results are initially analyzed, looking for outliers, or trying new analyses in order to obtain a p value of just under 0.05, which can lead to nonreplicable results

open science

the practice of sharing one’s data, hypothesis, and materials freely so others can collaborate, use, and verify the results

open data

when psychologists provide their full dataset on the internet so other researchers can reproduce the statistical results or even conduct new analyses on it

open materials

when psychologists provide their study’s full set of measures and manipulations on the internet so others can see the full design or conduct replication studies

preregistaration

a term referring to a study in which, before collecting any data, the researcher has stated publicly what the study’s outcome is expected to be

theory-testing mode

a researcher’s intent for a study, testing association claims or causal claims to investigate support for a theory

universality assumption

an explicit or implicit belief by researchers that all participants would act pretty much the same

cultural psychology

a subdiscipline of psychology concerned with how cultural settings shape a person’s thoughts, feelings, and behavior, and how these in turn shape cultural settings

lab study

a study that takes place in a standardized location, enabling researchers to control potentially extraneous factors

field research

(also field experiment) a real-world setting for a research study

ecological validity

(also mundane realism) the extent to which the tasks and manipulation of a study are similar to real-world contexts; an aspect of external validity

experimental realism

the extent to which a laboratory experiment is designed so that participants experience authentic emotions, motivations, and beahaviors