Research Methods Chp 13-14

quasi-experiments

studies in which the researcher lacks complete control

nudging people towards organ donation

Compared organ donation rates in countries that use presumed consent (opt-out) vs. countries that do not use presumed consent (opt-in)

psychological effects of cosmetic surgery

Compared self-esteem of people who underwent cosmetic surgery and people who only considered it

popular shows and suicide

Assessed the effect of the popular show 13 Reasons Why on suicide rates in the U.S.

Effects of legislation on opioid abuse

Assessed whether state laws could slow the opioid crisis in the U.S.

selection threats

when the kinds of participants at one level of the IV are systematically different from those at the other level

design confound

when some outside variable accidentally and systematically varies with the levels of the targeted IV

maturation threat

when an observed change in the DV could have emerged spontaneously over time

History threat

when an external, historical event happens for everyone in a study at the same time as the treatment

regression to the mean

when an extreme outcome is caused by a combination of random factors that are unlikely to happen in the same combination again, so the extreme outcome gets less extreme over time

attrition threat

when systematic kinds of people drop out of a study over time

some other threats to internal validity in quasi experiments

testing and instrumentation threats, observer bias, demand characteristics, and placebo effect

advantages of quasi experiments

high external validity, true experiments have ethical concerns, Quasi-IV usually has good construct validity, and present real-world opportunities for studying interesting phenomena and important events

quasi-experiments and correlational studies

neither use manipulated variables or random assignment

in quasi experiments, researchers

tend to select their samples more intentionally (ex. time period in 13 reasons why study)

replicable

study's results are replicable

direct replication

repeat original study as closely as possible to see whether same effect is obtained in new sample

conceptual replication

explore same research question but use different procedures

replication-plus-extension

replicate original study but add variables to test additional research questions

why might a study not be replicable?

conceptually sensitive effects, number of replication attempts, and problems with original study

conceptually sensitive effects

when the replication context is too different, replication is more likely to fail

number of replication attempts

any single study always has the potential to miss a true finding, leading to a failed replication

problems with original study

small samples can accidentally lead to significant findings that cannot be replicated because there probably was not a real effect in the first place

The most credible conclusions are based

on a body of evidence

meta analysis

type of statistical analysis that mathematically averages the results of all the studies that have tested the same variables to see what conclusion the whole body of evidence supports

file drawer problem

a meta-analysis might overestimate the true effect size because stronger relationships are more likely to be published than negligible/null effects (or opposite effects)

questionable research practices

p-hacking, underreporting null effects, HARKing, and using small samples

p-hacking

The use of data mining to uncover patterns in data that can be presented as statistically significant, without first devising a specific hypothesis as to the underlying causality.

HARKing

hypothesizing after results are known

using small samples

In a small sample, a few chance values can influence the data set, so the study's estimate is imprecise and less replicable

underreporting null effects

researchers mislead about the strength of the evidence by not reporting conditions or measures that did not support the hypothesis

External validity comes from

how participants were selected, not how many participants there were

Ecological validity

an aspect of external validity in which the focus is on whether a laboratory study generalizes to real- world settings

When a study takes place in the real world

it occurs in a field setting and has high external validity

experimental realism

Laboratory research can be just as realistic as research conducted in the real world

Experimenters create artificial situations that allow them to eliminate alternative explanations for the results

in these studies, internal validity is often prioritized over external validity