Research Methods and Statistics

Flashcards for EPPP exam review.

Research Design

A systematic approach to knowledge acquisition, ranging from simple observation to causal investigations.

Research Methods

The tools scientists use to acquire knowledge.

Theory

An organized set of beliefs about a phenomenon.

Hypotheses

Predictions about associations between variables, often derived from a larger theoretical framework.

Internal Validity

The extent to which the association between x and y is causal in nature.

Statistical Association

Occurs when the hypothesized cause and its effect covary.

Temporal Precedence

Occurs when the putative cause precedes its effect in time.

Nonspuriousness

Occurs when the hypothesized cause—and not some other factor—is responsible for the effect.

External Validity

The extent to which the causal association can be generalized across variations in study instances.

Statistical Conclusion Validity

Examines whether there is a statistical association between x and y and the magnitude of this association.

Construct Validity

The extent to which inferences can be made from particular study instances to the higher-order constructs from which they presumably derive.

Randomized Experiments

Considered the gold standard for assessing causality, involving random assignment to conditions.

Efficacy Trials

Intervention effects are examined under ideal circumstances.

Effectiveness Trials

Intervention effects are examined under real-world conditions.

Intent-to-Treat Analyses

Analyze outcome data from participants as a function of their original group assignment.

Single-Case Experiments

Designed to assess the causal influence of an intervention on an outcome with intensive assessment.

ABAB Designs

Single-case design that alternates the baseline (A) phase with an intervention (B) phase.

Multiple Baseline Designs

Replication of an effect is sought over multiple baselines.

Quasi-Experimental Studies

Experiments that lack random assignment of units to conditions.

Correlational Studies

Studies conducted when the researcher is not actively manipulating anything.

Case-Control Designs

Compare a group of participants who possess a certain characteristic with a group who do not.

Cohort Designs

An intact group is followed over time to examine the emergence of some outcome of interest.

Cross-Sequential Design

Multiple cohort studies where groups differ in age or developmental marker at the study’s start.

Uncontrolled Case Studies

Follow a single individual over time in the hope of understanding the case in a more comprehensive manner.

Maturation

Threat to validity when naturally occurring changes are mistaken for an intervention effect.

History

Threat to validity when some event occurs during the study and impacts the results.

Statistical Regression

Occurs when extreme scores tend to revert back to the mean on a subsequent evaluation.

Attrition

Threat to validity when the pattern of participant drop-out impacts the results.

Testing

Threat to validity when exposing individuals to the pretest changes them in ways that might be mistaken for an intervention effect.

Instrumentation

Threat to validity when the measurement tool changes and impacts the results.

Selection

Occurs in multiple-group studies when systematic differences among intervention groups can be mistaken for an intervention effect.

Levels of Evidence

Ordering evidence sources along a continuum from low to high based on internal and external validity.

Meta-Analysis

Quantitative syntheses of the literature, seeking to explain variability in study effect sizes.

Latent Constructs

Unobserved constructs of interest.

Observable Measures

Tests are considered fallible representations of latent constructs.

Nomological Network

The relations among observed measures, latent constructs, and relations between.

Classical Test Theory

Assumes that the variance of an observed measure comprises true score variance and random error variance.

Internal Consistency

Measure reliability on a single testing occasion by examining the degree of inter-item correlation.

Cronbach's Alpha

A common index of internal consistency.

Test-Retest Reliability

Measure the reliability of a measure over time.

Alternate-Forms Reliability

Assessed by correlating two tests meant to measure the same construct.

Inter-Rater Reliability

Computed when at least two raters are used to code observational data.

Validity

The extent to which a test measures what it purports to measure.

Face Validity

The extent to which items appear to measure the construct of interest.

Content Validity

Examines whether test items adequately represent the content domains for the relevant construct.

Structural Validity

The extent to which the structure of the measure is consistent with the theorized factor structure.

Criterion Validity

The measure should correlate with other measures in a manner consistent with a priori hypotheses.

Convergent Validity

Measures of constructs converge with either measures of similar constructs or different measures of the same construct.

Discriminant Validity

Measures of constructs diverge from measures of dissimilar constructs.

Generalizability Theory

Assumes that in addition to true score variance, there are additional possible sources of systematic variance.

Item Response Theory (IRT)

Extends Classical Test Theory by taking into account that an individual’s response is influenced by qualities of both the individual and the test item.

Exploratory Factor Analysis (EFA)

Used when developing/refining a measure or when researchers are less certain about the measure/construct’s factor structure.

Confirmatory Factor Analysis (CFA)

Used when existing theory makes specific predictions about a measure/construct’s factor structure.

Measurement Error

The non-depression-related variance.

Life Events Data

L in LOTS data.

Observational Data

O in LOTS data.

Testing Data

T in LOTS data.

Self-Report Data

S in LOTS data.

Nominal Scale

Used to categorize qualitative variables that cannot be ordered along a quantitative dimension.

Ordinal Scale

Allows researchers to arrange responses according to order or relative rank.

Interval Scale

Allows one to order and examine magnitude differences among responses with equal units, lacking a true zero point.

Ratio Scale

Variables measured on a ratio scale have both equal intervals and a true zero point.

Descriptive Statistics

Used to organize, describe, and simplify data.

Central Tendency

Used to identify the center of a distribution of scores.

Mean

The arithmetic average.

Median

The score that corresponds to the 50th percentile.

Mode

The most commonly occurring score.

Variability

Describe the scatter or dispersion of scores in a distribution.

Range

Computed by subtracting the minimum from the maximum observed value.

Interquartile Range

Captures the middle 50% of the distribution.

Standard Deviation (SD)

Captures the average distance of scores from the mean.

Variance

The square of the standard deviation.

Normal Distribution

Characterized by a distinct peak in the center and symmetrical halves.

z Scores

Provide a common scale and as such, the z distribution is often referred to as the standard normal distribution.

Inferential Statistics

Often divided into parametric and nonparametric approaches.

Parametric Statistics

Compared to nonparametric statistics, parametric statistics make more distributional assumptions.

Nonparametric Statistics

Typically used when assumptions of parametric approaches are violated.

Null Hypothesis Significance Testing (NHST)

The researcher specifies two mutually exclusive hypotheses (null and alternative) regarding the population parameter.

Type I Errors

Occur when a true null hypothesis is incorrectly rejected.

Type II Errors

Occur when a false null hypothesis is not rejected.

Statistical Power

The probability of correctly rejecting a false null hypothesis.

Clinical Significance

Goes beyond statistical significance to describe the clinical importance of an effect.

One-Sample z Test

Used to compare a single sample mean to a population mean, when the population standard deviation is known.

One-Sample t-Test

Used if the population standard deviation is unknown.

Independent Samples t-Test

Used to test mean differences between two populations on a continuous measure.

Paired Samples t-Test

Examines mean differences across observation pairs.

Analysis of Variance (ANOVA)

Model often used when predictor variables can be coded as finite categorical variables and the outcome is at the interval or ratio level.

Main Effects

Examine the unique effect of an independent variable on an outcome.

Interactions

Examine whether the effects of one predictor on an outcome vary significantly as a function of another predictor.

Analysis of Covariance (ANCOVA)

Model by adding one or more covariates.

Multivariate Analysis of Variance

Model allows for multiple dependent variables to be analyzed in a single model.

Pearson Product-Moment Correlation (r)

Measures linear associations between two continuous variables.

Ordinary Least Squares (OLS) Regression

Allows for the prediction of a single continuous outcome from one or more predictor variables.

Logistic Regression

Used when the outcome variable comprises ordered or unordered categories.

Path Analysis

Models can be structured to reflect a stronger causal ordering.

Structural Equation Modeling

Further extends path analysis to include path analysis with latent variables.

Chi-Square Test

Test the hypothesis that the data follows a specified distribution.

Mann-Whitney Test

Nonparametric alternative to the independent samples t-test.

Kruskal-Wallis Test

Nonparametric alternative to the between-group ANOVA.

Wilcoxon Signed Ranks Test

Nonparametric alternative to the paired samples t-test.