(from top) Psych Research Methods Second Exam

Categorical/Nominal

• Qualitative categories, e.g., drug types, emotions, elementary schools in a district

  • Grouping variables for t-tests, ANOVA, regression

  • Nonparametric Analyses (chi-square)

Ordinal

Ordered categories, like ranking

Continuous

Interval (standard difference between integers) or ratio scales (difference between integers with true zero)

Numerical scales – correlation & regression, outcome variables for t-tests, ANOVA, regression

Random Sampling

All members of a population have an equal chance of being selected to participate in a study

Probability

Stratified Sampling

A population is first divided into groups, or strata, and then members of each group are randomly sampled.

• Can be proportional or nonproportional – e.g., want to reflect population or over sample specific groups

Probability

Convenience Sampling

Participants are selected based on access to study

Not all members have the same (or any) chance of being selected for participation.

Non-Probability

Random Sampling Error

When sample does not match population due to chance

• Likelihood of this type of error decreases as the sample size increases

Sampling Bias

• When sample does not match population due to a design flaw

• Sampling strategy leads to disproportionate sample of some groups, potential exclusion of other groups, because not every member of the population has an equal chance of being selected

What is Replication crisis?

• 97% of original studies had statistically significant findings compared to 36% of replications

• If a study is unable to replicate, that could imply the findings do not generalize to new populations, contexts, or time

Why did the replication crisis happen?

• Small sample sizes – reduce reliability

• Inability to fully control for confounds, such as changes over time

• Publication bias – pressure to publish new data on new ideas, and focus on publishing significant results

• Some findings might not be stable

What has been done to correct replication crisis

• Sample sizes have increased

• More robust significance is associated with greater likelihood of replication (p < .01 vs. p < .01-.05), and journals have started to be more likely to require more robust significance

• Increased rigor in reporting – such as pre-registration: report study design and hypotheses prior to data collection to prevent p-hacking and increase transparency