cogs 14A post-miderm content

requirements of a between-groups design experiment (2)

• groups being compared must be equivalent before the independent variable is manipulated

• experimental design must be free of confounds

sampling bias

participants are chosen in such a way that they do not represent the population as a whole

• could affect the external validity of the experiment

sampling error

the mismatch between sample statistics and the population parameters they estimate

• will always be present

• occurs due to inherent variability in the population

• ensures that sample statistics will not match each other exactly

pre-screening

sometimes we ______ participants on important characteristics

• goal: create a more uniform sample that includes fewer extraneous variables

• limits ecological validity

selection bias + solution

occurs when the researcher makes non-random assignment which may lead to systematic differences, or if a participant selects the group they’re in

• large sample sizes help smooth out effects of some non-random assignment

solution: attempt a time-series design; try a matched, non-equivalent control group

experimenter bias

the researcher’s expectations (or interests) may affect how the experimenter interprets behavior in the experiment; influences measurement of the dependent variable

demand characteristics

the participant picks up some cue that leads them to perform differently, changing the outcome of the dependent variable

• participants may change their behavior to please or thwart the experimenter

• can use filler trials to distract participants, or give a complicated task + only measure the data of interest

single-blind procedure

either the participant or experimenter don’t know the condition they’re in

double-blind procedure

neither the participant nor the experimenter know the experimental conditions; this will limit any systematic bias in behavior or treatment

different treatment of groups

• the two groups must go through the same experimental procedure (protocol) for it to be a valid comparison

• instructions, setting, task, and other factors must match each other to avoid confounds

• different experimenters might work with different experimental groups, affecting performance in the groups differently

instrumentation effects

occur when the instrument used to record the dependent variable changes in accuracy or sensitivity over time

subject attrition + solution

people can leave in-progress experiments

• non-sytematic, systematic

solution: no solution :(

non-systematic attrition

reasons unrelated to the study; equally likely in each group

systematic attrition

reasons related to a specific condition; perhaps only strongly motivated people finish one condition

ceiling effect

all scores are near the top of the measurement scale

floor effect

all scores are near the bottom of the scale

between-groups design: advantages

• different groups of people are exposed to only one level of independent variable each; causal effects + comparisons are straight-forward

• participating in only one condition lessens chance of attrition

• may be the only option when impractical to run participants in multiple conditions

between-groups design: disadvantages

• for each condition (level of independent variable), you need an additional group of participants

• demand characteristics may be different for each group

• keeping the groups independent may lead to different treatment

• people in the groups may be very different; the subject variables may widely vary between groups; may become nuisance variables; adds error variance

primary characteristics of within-groups design

• all participants experience all levels of the independent variable; each participant is their own control

• reduces error variability by eliminating inherent differences in the subject variables

• fewer participants are required than between-groups designs; important for work with patients or expensive studies

• statistical tests subtract the common variability → decreases the chance of a Type II error

pretest-posttest design

participant performs a task, dependent measure is collected; treatment applied; participant performs the task again; the second measure is compared with the first

repeated-measures design

participants undergo all conditions (levels of independent variable), in any order; collect measures to each stimulus

mixed design

aka cross-over repeated measures design, includes a between-groups comparison (and a within-subjects design)

disadvantages of within-groups designs

• demand characteristics

• carryover effects

  • practice effects

  • fatigue effects

carryover effects

aka order effects, includes practice effects + fatigue effects

practice effects

as you perform a task more, you get better at it. early trials may be worse than later trials, especially on hard tasks.

fatigue effects

as participants perform a boring task, they may lose attention & get progressively worse

linear practice effects

improvement changes gradually over time; to control for this, you would need to counterbalance your conditions.

non-linear practice effects

improvement changes quickly at the beginning as participants learn the task; may flatten; can be controlled by using practice trials before starting the main experiment

history effects + solution

experiments that last several days can be interrupted by life events outside of the experiment

• coincidental, but can be a real problem if repeated measures are done in a specific order

solution: time-series design

maturation effects + solution

people may change their behavior over time due to physical changes (e.g. growth, learning, getting hungry, etc.)

solution: similar, nonequivalent control group; time-series design

testing effect

participants may get better scores bc they are used to the experiment + task through habituation

solution: similar, nonequivalent control group; time-series design

counterbalancing

to lessen the impact of some carryover effects + time-related confounds, you can balance the order in which different conditions are presented to different participants

complete within-subjects design

typically used for 2-3 levels of independent variable. all possible orders of conditions are presented to participants.

incomplete within-subjects design

typically used for more than 3 levels of independent variable or studies with lots of possible stimuli. each participant receives a unique order, but does not receive all possible orders of conditions

pre-experimental design

• don’t/can’t randomly assign people to groups; may be no control group.

• people self-select.

• e.g. can use a simple pretest-posttest comparison of the mortality rate before + after the program was instituted.

regression toward the mean + solution

if the first measure was extreme, the next measure is likely to be less extreme (closer to the mean)

solution: time-series design, matched non-equivalent control group

distinguishing true experiments, pre-experiments, quasi-experiments

quasi-experimental designs

allow you to compare the result of a manipulation of the independent variable with a different group of people

• control group should be similar, but assignment to the treatment & control groups was not random

single time-series design

several repeated measures taken before + after the independent variable is manipulated

• many factors are the same at each time point

• no control group present

multiple time-series design

several measures taken before + after the independent variable is manipulated

• similar control group presented for comparison

• provides more supporting evidence that the treatment had an effect

• lacks random assignment → can’t make causal claims

interaction of selection bias with other confound + solution

in non-equivalent control group design, perhaps one group is more affected by a confound than the other

solution: multiple time-series

factorial design

an experimental design that simultaneously examines the effects of 2+ independent variables (factors) on a dependent variable

null (H₀) vs. alternate (H₁) hypotheses for a described experiment

• null (H₀): μ₁ = μ₂ = μ₃ = μ_n

• H₁: The null [any of the nulls] is false.

one-factor ANOVA

one independent variable

main effect

significant difference between groups

• average the value of two levels + see if they differ

interactions

condition affects dependent variable differently in different conditions/groups

• parallel lines indicate no interaction

factorial design (e.g. 2×2)

factorial designs > multiple comparisons of two levels of a single independent variable

when you run more statistical tests, you risk a false alarm each time; more tests → more false alarms

how adding a subject variable (e.g. sex) in factorial designs can be very important to basic science

• explicit between-groups comparisons using a subject variable can find potential differences

• avoids missed opportunities to find differences that may have important clinical effects

longitudinal studies

collect data at multiple points in time (within-subjects design). the goal is to observe patterns of change over a long period of development [time as an independent variable]

cross-sectional studies

collect data at one point in time. data may come from many different age groups (between-groups design). excellent for exploratory + descriptive studies [age as a subject variable]

advantages of cross-sectional studies

• cost-efficient

• fewer logistics problems

disadvantages of cross-sectional studies

• can only find differences in groups; no causal explanations

• cohort effects may emerge: differences due to how the age groups developed separately, not in how they aged

cohort effects

differences due to how the age groups developed separately, not in how they aged

advantages of longitudinal studies

• understanding causal processes

• more in-depth understanding of complex behaviors + processes

disadvantages of longitudinal studies

• resource intensive

• logistically difficult

• problem of attrition

trend studies

examines changes within the general population over time; e.g. census studies + polls

cohort studies

examines a more specific population as those individuals change over time; within-subjects designs

prospective research

begin with a sample + follow them over time

retrospective research

• may go through old records + reconstruct activities of cohorts over time

• could also ask participants to recall their past (more susceptible to subjective errors)

how longitudinal studies are more useful for evaluating clinical interventions in a population

• many different dependent measures can be measured over time

• may suggest interventions to improve performance or delay onset of age-related pathologies

most aging studies are _____

cross-sectional

traits of research in natural settings

• lack of control often requires using the descriptive method

• researchers observe participants in natural environment using many methods

• surprising results may help generate new hypotheses

• very high ecological validity; can be used to confirm results from lab

traits of research in laboratory settings

• precise control over variables using the experimental method

• groups can be compared on the same task

• artificial tasks can be used to probe specific causal relationships

• lower ecological validity

naturalistic observations

researcher remains unobtrusive

• covert

• overt

  • desensitization

  • habituation

covert operations

allow for the least amount of potential disruptions

overt observations

require great care in order to not change the nature of the monitored behavior

• desensitization

• habituation

desensitization

researcher gets progressively close to participants; decrease fight-or-flight response

habituation

repeated exposure used until presence no longer affects behavior at all

participant observations

researcher becomes part of the group being observed

• disguised

• undisguised

disguised participant observations

subjects are blind to the researcher’s role

undisguised participant studies

researcher joins the group studied, but is recognized as an outsider

• ethnography falls into this category

traits + advantages + disadvantages of a field experiment

• researcher studies the phenomenon of interest in ‘real life’ scenarios

• often the only way to get the information

• allows for causal inference + ecological validity

• problems of internal validity

potential problems with observational studies done in natural settings

1. ethical issues over lack of free, informed consent; may be issues with potential harm

2. observational studies (except field experiments) don’t allow causal inferences; strictly correlational

3. presence of observers may influence behavior, distorting ecological validity

4. observer may make biased observations

Rosenhan pseudopatient experiment

mental hospital disguised participant study; subject to bias…

Hawthorne effect

participants may change their behavior if they know they’re being observed

• reactivity in the participant’s behavior may be due to demand characteristics (guessing what the observer wants) or hiding socially sensitive responses. threatens internal validity of the approach

expectacy effects

conscious or unconscious beliefs + preferences that may affect how dependent variables are recorded + interpreted

ecological validity

naturalistic observation > participant observation > field experiment > lab experiment

control of extraneous variables

lab experiment > field experiment > naturalistic + participant observation

interobserver reliability

the degree of consistency obtained between observers

narrative records

tend to be more qualitative + have a broader scope

• no restrictions on pre-defined types of behaviors; good when looking for new or spontaneous observations (generating hypotheses)

• often need some type of data reduction (coding)

checklists

focus on more specific behaviors that have been operationally defined

• focus on a small set of behaviors; good for gathering data in a field experiment (evaluating a hypothesis)

• don’t require as much data reduction, but are limited in what they can measure

technology-mediated advanced methods

may use specific devices or techniques developed for the specific research issue

• may augment use of narrative records (or checklists)

• primarily uses specialized technology to capture something specific about the situation

• may also use technology to trigger + record certain behaviors; can be longitudinal if you track users over time

time sampling

take observational measurements at systematic or random times that are most representative of the group being studied

• best for continuous behaviors

event sampling

random or systematic sampling of events that include specific behaviors of interest

situational sampling

observations of an operationally defined situation are made in different settings + circumstances

• note how the situation can occur in a number of places

reactive measures

procedures where taking measurements may influence a participant’s behavior

non-reactive measures

collected in the absence of the participant + should have no effect on behavior that already happened

• can be done in physical trace studies or archival data investigations

physical trace studies

look at physical evidence left behind as a result of behaviors or activities

• accretion measures

• erosion measures

accretion measures

examine the build-up of material over time that results from a behavior

erosion measures

based on the wearing down of material through use

natural trace measures

occur without any action being taken by the researcher

controlled trace measures

are put in place by the researcher to specifically measure some activity

• e.g. installing a carpet that wears down easily

selective survival

not all traces/behaviors may be equally represented over time

• traces or products may degrade over time; we only see what survived

selective deposit

different populations may be more or less likely to leave behind traces

• e.g. campsite use: need to note that more experienced campers leave behind fewer traces

archival data study

previously recorded information is the focus of analysis

continuous (running) records

items that are updated on a regular or scheduled basis

discontinuous records

sporadic or unique

records

items that were intended for the use of others at the time of creation

documents

items that were generally intended for personal use only. typically not considered public documents