research methods exam 2

Comprehensive vocabulary flashcards covering experimental design controls, factorial models, replicability, single-case methodologies, mediation, moderation, qualitative analysis techniques, and mixed methods research designs.

Psychological Experiment

• there is a specified independent variable that is manipulated and a specified dependent variable. we are trying to understand how the manipulation of the IV influences the DV

• controlled situation in which one or more factors are varied and the others are kept constant

Independent Variable (IV)

The predictor variable in an experiment that is systematically manipulated by the researcher

Dependent Variable (DV)

The outcome or criterion variable in an experiment that is measured to determine the influence of the manipulated independent variable

Conditions for Establishing Causality

1) the independent variable is related to the dependent variable

2) changes in the independent variable occur before the dependent variable changes (temporal order)

3) no plausible alternative explanation exists for the relationship

Random Assignment

A control procedure that uses probability to create groups by ensuring that every participant has an equal chance of being placed in any condition, so extraneous variables are distributed equally and not affecting outcomes

best when participants are randomly sampled from a population

when do we randomize?

once all sampling is complete

how to randomize?

randomization software or online generators

Quasi-Experimental Design

A research design used when participants cannot be randomly assigned to groups, but are instead separated into conditions in a pre-existing or meaningful way

experimental design lies on a continuum

weak - moderate - strong

goal of experimental design

maximize validity/meet condition 3

aka determine the outcome is due to your manipulation not another outside influence

procedure 1: holding variables constant

A control procedure where a pre-determined extraneous variable is kept the same across all participants, often implemented at the sampling level

procedure 1: holding variables constant EXAMPLE

procedure 2: blocking

A matching procedure where a potential extraneous variable is built directly into the research design by creating discrete levels or blocks and randomly assigning participants to conditions within each block

procedure 2: blocking EXAMPLE

• testing a new medication for depression

• create blocks for mild, moderate, and severe depression

• randomly assign to placebo or medication group within each severity group

• make comparisons of how medication works depending on depression severity

why might we use procedure 2 (blocking)?

• findings are more specific

• moderating effect

• eliminate confounds specific to each group

• we can make comparisons between groups

• IV might work differently for each group

• equal/even distribution

procedure 3: Yoked Control Condition

A matching procedure where each control participant is paired with an experimental participant to undergo the protocol simultaneously; the active participant makes choices or puts in effort, while the yoked participant receives identical outcomes or rewards regardless of effort

procedure 3: Yoked Control Condition EXAMPLE

two groups hear an ongoing loud noise that suddenly stops, but one group has a button in front of them (the noise stops when the experimental group pushes the button). the researchers could then explore if there are psychological differences when someone perceives they were the reason the noise stopped.

why use a yoked control?

• helpful for studying social phenomenon, reinforcers, etc

• power of control, choice, effort

• controls for treats such as attention or contact from experimenter

procedure 4: Precision Control

A sophisticated matching technique where participants are matched in exact pairs across multiple specific variables, such as age, gender, and race.

Isolation of IV effects are based on very specific variables, making the intervention and control group as identical as possible, to reduce the chance of these variables impacting the study

procedure 4: Precision Control EXAMPLE

I want to test a depression medication, so I match each participant from the experimental group with one from the placebo group based on socioeconomic status, place they live, age, gender, and ethnicity to control for the effect these variables might have on someone’s depression symptoms

procedure 5: Frequency Distribution Control

A matching technique that equates overall participant groups on a given variable by matching their summary distributions (similar means and standard deviations) rather than matching individual pairs

procedure 5: Frequency Distribution Control EXAMPLE

Counterbalancing

The process of varying the presentation order of experimental stimuli or conditions across participants to control for order effects and carryover effects.

if you are worried the order in which you present info will impact outcomes, you can___ to mitigate this effect

Order Effects

Differences in participant outcomes or performance that occur as a direct result of the sequence in which conditions or stimuli are administered

Carryover Effects

A threat to validity where learning, fatigue, or lingering exposure from one treatment condition influences participant performance in a subsequent phase or condition.

randomized counterbalancing

for each participant, sequence of stimuli is randomly determined

ex: 123, 132, 213, 231, 321, 312

Intrasubject Counterbalancing

A counterbalancing technique in which the sequence of stimuli is administered to each individual participant in more than one order (e.g., presenting conditions in order 1, 2, 3, and then 1, 3, 2).

group counterbalancing

administering different sequences to different groups

complete group counterbalancing

participant groups receive ALL of the possible sequences

incomplete group counterbalancing

participant groups receive SOME of the possible sequences

Double-Blind Experiment

An experimental control procedure in which neither the participants nor the experimenters interacting with them are aware of condition assignments, preventing demand characteristics (participants trying to conform to expected outcomes) and experimenter expectancies (experimenter bias)

gold standard

partial blind experiment

participants do not know their group

Between-Subjects Design

An experimental design in which two or more separate groups of participants are tested and compared across different levels of the independent variable

Within-Subjects Design

A repeated-measures design in which a single group of participants is exposed to all conditions or tested across multiple timepoints, serving as their own control

Factorial Design

A research design that includes two or more independent variables (factors), allowing researchers to simultaneously test main effects and interaction effects.

covariates

a variable that is potentially predictive of or related to an outcome (dependent variable) in a statistical study, and is included in a model to control for its effect

Mixed Design

A factorial study design that incorporates at least one between-subjects factor and at least one within-subjects factor.

Main Effect

The isolated, overall effect of a single independent variable on the dependent variable, averaged across all levels of all other independent variables in a factorial design.

Interaction Effect

An effect occurring when the relationship between an independent variable and the dependent variable changes depending on the level of another independent variable, visually depicted by non-parallel lines on an interaction plot.

Covariate

A variable on which experimental groups differ significantly at baseline that is statistically controlled for in analysis to ensure valid outcome comparisons.

between subjects design EXAMPLE

group A: animal therapy + CBT

group B: cbt only

hypothesis: animal therapy + CBT will yield lower anxiety symptoms than CBT alone

within subjects design EXAMPLE

assess baseline anxiety → give CBT → assess anxiety → give animal therapy → anxiety symptoms will decrease overtime

mixed/factorial design example

CBT + Animal therapy group will have the lowest anxiety scores

assess before and after of CBT alone

assess before and after of CBT + animal therapy

big picture takeaways from Replicability paper

• most studies in psychology can’t be replicated

• pushing out papers to meet a quota/ keep job is not real research

• confirmation bias in research skews outcomes

• psych as a field needs to work on being transperant and not overstating findings

replicability

the ability to repeat a study and obtain the same results

crucial for establishing the reliability and validity of findings, as many phenomena cannot be studied directly and/or with objective markers; helps us build confidence in research

high replicability

• strengthens the credibility of psychological research

• ensures findings are robust

• builds a foundation for theory development

Direct Replication

Repeating a study using the identical methods, procedures, and materials to verify the original findings; a verification of the original study results

Conceptual Replication

Repeating a study while altering specific operational definitions, variables, or participant populations to determine if the conceptual findings extend to new contexts.

similar enough to OG to draw parallels

factors contributing to low replicability

• small sample sizes (leads to false positive/type 1 error, or not enough power to determine effects- false negative/ type 2 error)

• questionable research practices: p hacking, cherry picking data

• publication bias: preference for publishing positive results over null findings, so researchers do things like removing outliers to get significant results

• “file drawer effect”: null studies are not published and go back into the “file”

reproducibility crisis

many psychological studies have failed to replicate, raising concerns about the reliability of research findings

ways to increase replicability

• preregistration of studies

• increasing sample sizes

• open science practices

Pre-Registration of studies

The practice of publicly registering study hypotheses, methodology, and analytic plans before beginning data collection to reduce flexibility in analysis and avoid p-hacking.

say your stuff ahead of time, be held accountable when you find results that are null,etc

cant switch up

increasing sample sizes

larger samples enhance statistical power and reduce the likelihood of type 1 errors

open science practices

encouraging data sharing, materials sharing, and transparency in methodologies to allow other researchers to verify findings; give everyone access to data

ex: provide the exact ?s you used in a questionnaire

increasing the methodological rigor

• use diverse samples- ensures findings are generalizable across populations and settings

• replication studies- actively conducting direct and conceptual replications of significant studies to test the reliability of findings

• robust statistical methods- employing techniques like bayesian statistics and hierarchical modeling to account for variability and enhance interpretation

how can our field move towards a more honest future?

• encourage collaboration: building networks among researchers to share ideas, data, and findings to promote transparency and replication

• recognizing and rewarding replications: journals and funding agencies should value and support replication studies to combat the publication bias against null results

• educating researchers: training in rigorous research practices and statistics to promote high quality study designs

Single-Case Research Design

A specialized within-subjects quasi-experimental design that investigates the effects of a treatment condition on an individual participant or a single small group across time

technically quasi because group is not randomly assigned

time series design

measuring a variable/trait/factor across a series of time points

time series design EXAMPLE

if a therapist wanted to test how implementing daily mindfulness activities would improve the anxiety of a client. The therapist could assign their client two activities to practice every day, then assess their anxiety once a week in sessions for eight weeks

Interrupted Time-Series Design

implements an intervention within a period of time, assessing data before and after the intervention occurs

Interrupted Time-Series Design EXAMPLE

if a therapist wanted to see how a day-long meditation retreat would affect the anxiety levels in a client. The therapist could measure the anxiety in their client one week before and one week after the meditation retreat.

ABA Design

A single-case design where target behavior is recorded during baseline ($A$), during an intervention phase ($B$), and after withdrawing the intervention to return to baseline ($A$) to demonstrate reversal.

ABAB Design

An extension of the ABA design that reinstates the intervention in a fourth phase ($B$), utilized for ethical purposes to ensure the participant retains the beneficial treatment and to examine carryover effects.

ABAB design EXAMPLE

I have been trying to spend less time on social media and spend more time being productive or doing activities that make me feel more fulfilled.

A is my baseline of spending a few hours on social media every day.

B is my intervention of deleting these apps off my phone, and instead using chunks of free time to do activities such as study, meditate, journal, or read for fun.

A would be redownloading these apps in feeling I want to check in with them again and then spending time on them again

B delete the apps again, trying to increase the number of days I go without redownloading the apps and engaging in my preferred activities.

Multiple-Baseline Design

A single-case design where an intervention is sequentially administered across several participants, behaviors, or settings at staggered timepoints

Multiple-Baseline Design EXAMPLE

a therapist wanted to see how implementing mindfulness practices would affect anxiety for

multiple clients but wanted to control for clients who had been in therapy longer. The therapist

could assign the mindfulness practices to each client on their third week of therapy, then

assessing anxiety symptoms each week for eight weeks.

Changing-Criterion Design

A single-case experimental design where a participant's behavior is shaped gradually by changing the criterion (outcome/ determinant ) of success across successive treatment periods (adding onto to requirements to receive reward)

Changing-Criterion Design EXAMPLE

I wanted to increase my concentration for studying for long periods of time. I could start with ten minutes, then take a break. Then, I could study for twenty minutes before taking a break. Then for forty minutes, and so on increasing time studying between breaks to get to a point where I can keep my concentration for an hour or longer.

considerations for single case design

• establishing a baseline

• systematic changing of independent variables ( be thoughtful , account for their individual factors)

• length of treatment phases (amount of time you implement may effect outcomes)

• evaluating change (is it clinical meaningful, lasting,etc. this is observed, not statistical)

Mediator Variable

A third variable that accounts for the underlying mechanism explaining why or how an independent variable causes changes in a dependent variable.

a → __ → b

meditator EXAMPLE

if researchers found that daily meditation led to lowered anxiety levels, but the reason for the lowered anxiety was because meditation produced better emotional regulation. Emotional regulation would be the mediator.

Moderator Variable

A third variable that alters the strength or direction of the relationship between an independent and a dependent variable, tested statistically via an interaction term.

moderator EXAMPLE

if researchers found that those who meditated more often had lowered stress levels, but a moderator was socioeconomic status. Higher SES allows people more time to meditate and has an influence on lower stress levels.

Complete Mediation

A condition in mediation analysis where the direct relationship between predictor and outcome becomes zero when accounting for the mediating variable, meaning the mediator explains the entire relationship.

Partial Mediation

A condition in mediation analysis where the direct relationship between predictor and outcome is reduced but remains non-zero when controlling for the mediator, indicating that other mechanisms also exist.

Indirect Effect

The pathway in a mediation model combining path $a$ (predictor to mediator) and path $b$ (mediator to outcome), commonly evaluated for statistical significance using the product of coefficients method.

Qualitative Research

A research approach that investigates concepts and phenomena without quantitative data or statistical testing, emphasizing subjectivity, lived experience (phenomenology), and meaning-making.

pros of qualitative research

• capture rich ideas and info

• capture nuance

• more interpersonal, more connectedness to participants

• can be a more creative process

• more flexibility

• people aren’t put in boxes

cons of qualitative research

• lack of #s may make it difficult to determine what is significant

• subjective

• time consuming

Researcher Reflexivity

A qualitative researcher's active practice of self-awareness and transparent acknowledgment of how their own contexts, beliefs, and biases influence the research process and data interpretation.

Content Analysis

A qualitative analysis technique that applies a quantitative lens by assigning codes, tallying code frequencies, and interpreting findings based on how often specific codes appear.

Thematic Analysis

An interpretive qualitative analysis technique that uses coding procedures to identify, analyze, and interpret broad patterns of meaning (themes) across a dataset.

Grounded Theory

An exploratory qualitative methodology that develops an inductive theory from the ground up through continuous, simultaneous data collection and iterative coding.

theory emerges from the research

Interpretative Phenomenological Analysis (IPA)

A qualitative research approach focused on exploring how individuals make sense of their personal lived experiences regarding a specific shared phenomenon.

Narrative Analysis

A qualitative method that examines how individuals and small groups construct and tell personal stories, focusing on narrative structure, characters, tone, and context.

Discourse Analysis

A qualitative technique that analyzes language, verbal communication, and non-verbal gestures to examine power dynamics, social meaning, and larger societal structures.

Data Saturation

The phase in qualitative data collection where gathering additional data no longer reveals new properties, perspectives, or codes.

collected enough

Triangulation

The practice of using multiple methods, data sources, or analytical frameworks concurrently to enhance credibility and corroborate conclusions.

Explanatory Mixed Methods Design

A sequential mixed methods approach ($\text{QUANT} \rightarrow \text{QUAL}$) in which quantitative data are collected first and followed up with qualitative data to explain and elaborate upon the quantitative findings.

Exploratory Mixed Methods Design

A sequential mixed methods approach ($\text{QUAL} \rightarrow \text{QUANT}$) where qualitative research is conducted first to develop hypotheses, measures, or frameworks that are then tested quantitatively.

Convergent Mixed Methods Design

A mixed methods design ($\text{QUANT} / \text{QUAL}$) where quantitative and qualitative data are collected concurrently and integrated to evaluate how both sources of data converge to answer a research question.