Research Design Midterm

Sample vs Population

A sample is drawn from a population to estimate parameters and test hypotheses

Research Question

Asks if there is a difference or relationship; two answers: no difference (H₀) or difference (H₁)

Hypothesis Tested

The null hypothesis (H₀) is tested; rejecting H₀ supports the alternative hypothesis (H₁)

Probabilistic Logic of Hypothesis Testing

Uses probability to decide if sample results are unlikely under H₀

Alpha Level

Sets the probability threshold for a Type I error (e.g., α = .05)

Critical Region

Range of values for which H₀ is rejected; determined by alpha level

Statistic

The test (e.g., t, F) that compares sample data to what is expected if H₀ is true

Test Statistic

Value produced by the statistic; compared to critical value to decide on H₀

Decision Rule

If test statistic falls in the critical region, reject H₀; otherwise fail to reject

Type I Error

Rejecting a true H₀ (false positive)

Type II Error

Failing to reject a false H₀ (false negative)

Bradford-Hill Criteria

Standards for inferring causality in research

Strength of Association

Stronger relationships provide stronger causal evidence

Consistency

Findings replicate across studies

Specificity

Effect is linked to a specific cause

Temporality

Cause precedes the effect

Biological Gradient

Dose-response relationship supports causality

Plausibility

Causal claim fits current scientific knowledge

Coherence

Does not conflict with established facts

Experiment

Experimental evidence strengthens causal inference

Analogy

Similar known causal relationships support causality

Contextual Model

Psychotherapy is a socially situated healing practice focusing on relational and cultural factors

Therapeutic Relationship

Strong alliance between client and therapist

Expectations & Hope

Client belief in therapy's effectiveness promotes change

Rituals & Healing Context

Structured therapeutic activities that provide meaning and promote healing

Common Factors

Shared therapy elements (empathy, alliance, positive regard) that drive outcomes across approaches

Random Sampling

Selecting participants from a population to generalize findings

Random Assignment

Assigning participants to groups to control for confounds and improve internal validity

External Validity

Random sampling enhances generalizability

Internal Validity

Random assignment enhances causal inference

WEIRD Participants

Western, Educated, Industrialized, Rich, Democratic; overrepresented in research samples

Selection Bias

Threat to validity controlled by random assignment

Group Equivalence

Random assignment increases equivalence but does not guarantee it

Ensuring Group Equivalence

Use matching or statistical control (e.g., ANCOVA)

Matching Variables

Match on variables theoretically related to the dependent variable

Experimental Design

Random assignment used; allows for stronger causal conclusions

Quasi-Experimental Design

No random assignment; uses existing groups

Meta-Analysis

A quantitative review combining results from multiple studies using effect sizes

Effect Size

Standardized measure of the magnitude of a treatment effect

Medical Model

Assumes specific therapeutic ingredients cause change

Contextual Model

Assumes common factors and the therapeutic relationship drive change

No-Treatment Control

Group receives no intervention; measures natural change

Wait-List Control

Group receives treatment after a delay; ethical alternative

Placebo Control

Group receives inactive treatment mimicking real therapy

Attention-Only Control

Group receives therapist attention but no active treatment

Treatment-as-Usual (TAU)

Group continues standard care available in real-world settings

Nonequivalent Control

Used when random assignment isn't possible; helps rule out rival explanations

Alternative Treatment

Participants receive another active intervention for comparison

Ethical Concerns

Denying or delaying treatment raises ethical issues

TAU Strengths

Ethically sound and ecologically valid

TAU Weaknesses

Variability makes it hard to isolate effects

Deliberations When Selecting Groups

Ethics, scientific validity, feasibility, and generalizability

Intervention Package Strategy

Tests entire treatment program as a whole

Comparative Intervention Strategy

Compares two or more active interventions

Dismantling Strategy

Examines which treatment components produce effects

Constructive Strategy

Adds new treatment elements to enhance efficacy

Psychosocial Strategy 1

Base interventions on theory and research

Psychosocial Strategy 2

Adapt interventions to cultural and contextual factors

Psychosocial Strategy 3

Ensure replication and treatment fidelity

Absolute Efficacy

Overall evidence showing psychotherapy works compared to no treatment

General Efficacy

Synonymous with absolute efficacy; demonstrates psychotherapy benefits across studies

Four Major Classes of Validity

Internal, External, Construct, Data-evaluation

validity

Internal Validity

The degree to which results are due to the independent variable rather than confounds (e.g., history, maturation)

External Validity

The degree to which results generalize to other people, settings, and times

Construct Validity

The extent to which a test or manipulation truly represents the construct of interest

Data-evaluation Validity

The facets of the evaluation that influence the conclusions we reach about the experimental condition and its effect

What is a construct?

An abstract concept used to explain behavior or phenomena (e.g., depression, intelligence, self-esteem)

How is "construct" used?

Operationalized into measurable indicators, allowing abstract ideas to be studied empirically

Confound

An extraneous variable that covaries with the independent variable, making results hard to interpret

Wine Example (Kazdin)

Wine + social interaction co-vary; unclear which causes the effect, threatening construct validity

Alpha (α)

The significance level; the set probability of making a Type I error

Type I Error

Rejecting the null hypothesis when it is actually true (false positive)

Effect Size

A measure of the magnitude of an effect or relationship, beyond statistical significance

Why Effect Size Matters

Large samples can produce significance for trivial effects; effect size shows practical importance

Error Variance

Random variability not due to the IV; caused by measurement error, participant variability, environment

Error Variance & Effect Size

Higher error variance lowers effect size, making effects harder to detect

Statistical Power

Probability of correctly rejecting a false null hypothesis

Low Statistical Power

Increases the risk of a Type II error (false negative)

Sample Size & Power

Larger samples increase statistical power and reduce Type II error risk

Components of the Medical Model

1) Illness or Disease

2) Biological Explanation

3) Mechanism of Change

4) Therapeutic Procedures

5) Specificity

What is the first component of the Medical Model?

Illness or Disease - Disorder is identified through symptoms, history, and tests; may also include prevention (e.g., vaccines)

What is the second component of the Medical Model?

Biological Explanation - Each illness has a biological cause (e.g., influenza virus, H. pylori for ulcers)

What is the third component of the Medical Model?

Mechanism of Change - Treatment must target the biological system causing the disorder (e.g., neutralizing acid vs. antibiotics for ulcers)

What is the fourth component of the Medical Model?

Therapeutic Procedures - Treatment is based on explanation and mechanism, such as drugs, surgery, or procedures

What is the fifth component of the Medical Model?

Specificity - Treatment must work through its intended biological mechanism, proven by being more effective than placebo and linked to causal change

What is a placebo?

A placebo is a substance that has no active pharmacological properties that would be expected to produce change for the problem to which it is applied.

Major Threats to Construct Validity

1) Attention and Contact Accorded the Client

2) Single Operations and Narrow Stimulus Sampling

3) Experimenter Expectancies

4) Demand Characteristics

Attention and Contact Accorded the Client

The possibility that positive effects are due to extra attention given to participants, not the intervention itself

Single Operations and Narrow Stimulus Sampling

Results may reflect one experimenter, stimulus, or condition rather than the construct; multiple sources help rule this out

Experimenter Expectancies

Unintentional cues from experimenter (tone, expressions, instructions) influence participant responses

Demand Characteristics

Incidental cues in the study that "prompt" participants to act in ways mistaken for the effect of the IV

What are the three training models in clinical psychology?

Scientist-Practitioner (Boulder, 1949), Practitioner-Scholar (Vail, 1973), Clinical-Scientist (1991).

Which training model does APU PsyD align with?

Practitioner-Scholar Model (Vail Model).

What is the main goal of experiments?

To draw causal inference by testing relationships between variables and establishing internal and external validity.

What are the five critical components of an experiment?

(1) Treatment condition (IV), (2) Comparison condition (control/alternative), (3) Units of assignment (sample), (4) Random assignment, (5) Outcome measure (DV).

What is the difference between an independent and dependent variable?

IV = manipulated condition/treatment; DV = measured outcome.

What are the four categories of experimental validity (Kazdin)?

Internal validity, External validity, Construct validity, Data evaluation validity.

Define internal validity.

The degree to which a study establishes a trustworthy cause-and-effect relationship between a treatment and outcome.

Define external validity.

The degree to which study results can be generalized to other settings, populations, or times.

What are the eight threats to internal validity (Kazdin)?

History, Maturation, Testing, Instrumentation, Statistical regression, Selection bias, Attrition, Diffusion of treatment