research exam 2: wk 7 content

intro to quantitative research, experimental vs quasi-experimental designs

measures of central tendency (3) in descriptive statistics (& the purpose)

describe the typical value of the dataset

mean, median, mode

measures of variability/spread in descriptive statistics (3)

** describe how spread out or clustered the data is

range: difference between highest & lowest values

standard deviation: avg distance of scores from mean

variance: SD squared

descriptive statistics purpose

summarize the data

** think central tendency & spread

inferential statistics

allows researchers to draw conclusions about a population based on sample data; used to test hypothesis

** infer or generalize conclusions from data

t-test

inferential test compares means of 2 groups

ANOVA

inferential test compares means of 3+ groups

Chi-square

inferential test examines relationships between categorical variables

Correlation (Pearson’s r)

inferential test examines the strength/direction of a relationship between 2 variables

null hypothesis (Ho)

NO relationship or difference between variables

** researchers use statistical tests to decide whether to REJECT or FAIL TO REJECT Ho

alternative hypothesis (Ha)

there is a relationship or difference between variables

is the p-value statistically significant?

D. reject the null hypothesis

rationale: p-value is .03 which is less than alpha (.05) thus reject Ho
if p-value is less than or equal to .05 (alpha) → results are statistically significant → reject Ho
if p-value > a → fail to reject Ho

what is the independent variable

D. the new ambulation protocol

purpose of operational definition

B. make the concept measurable and consistent across the study

what is the level of measurment?

C

SATA

A, B, C

B, C, D

A

3 defining features of true experimental designs

1. Randomization: every subject has an equal chance of being assigned to any group; distrubutes known & unknown cofounders evenly

2. Control: control/comparison group is used to evaulate the effect of the IV

3. Manipulation: researcher actively manipulates the IV

experimental design notations standard symbols:

R

X

O

R = random assignment to groups

X = treatment/intervention (manipulation of IV)

O = observation/measurement (pretest or posttest)

pretest-posttest control group design

R O2 X O2 - experimental group

R O1 (no intervention) O2 - control group

posttest-only control group design

R X O

R (no intervention) O

explain solomon four-group design (purpose)

CONTROLS FOR TESTING EFFECTS

most rigorous true experimental design; resource-intensive, rarely used in nursing research

posstest-only control group design

R X O - experimental group

R O control group

subjects randomized to groups, NO pretest given

random controlled trial (RCT)

a true experimental design, gold standard for testing intervention effectiveness (STRONGEST evidence for cause-and-effect relationships, Level II)

strengths of true experimental designs (5)

• strongest design for establishing cause-and-effect

• randomized controls for known and unknown confounders

• highest level of quantitative evidence (Level II)

• findings support strong clinical recommendations

• replicable methodology

limitations of true experimental designs (5)

• may not be ethical or feasible (cannot randomize all variables)

• highly controlled settings may reduce real-world applicability (external validity)

• costly & time-intensive

• Hawthorne effect: subjects may alter behavior because they know they’re being studied

• attrition can compromise randomization over time

quasi-experimental designs

shares goal of testing cause-and-effect BUT lacks 1+ features of a true experiment; Level III evidence, weaker than RCT but stronger than non-experimental designs

used when…

• randomization is not ethical

• randomization is not feasible

• natural experiments

time-series design

a single group is measures repeatedly before and after an intervention

PURPOSE: multiple measurements before intervention establish a stable BASELINE TREND, multiple measurements after intervention show there the trend CHANGES

repeated-measures design

same subjects are measured multiple times under different conditions

• each subject serves as their own control → reduces variability d/t individual differences

crossover design - what is it, key limitation, useful for?

subjects receive ALL interventions being studied but in different sequences/order

• washout period: a time interval between treatments to allow effects of the first treatment to dissipate

• carryover effect: when the effect of the first treatment persists and influences the second treatment’s results (KEY LIMITATION)

useful for studying interventions w/ REVERSIBLE, short-term effects

one-group pretest-posttest design - what is it, what is it missing

O1 X O2 - a single group is measured, given the intervention, measured again

• weakest design b/c NO control group

• vulnerable to history, maturation, testing effects (weak internal validity)

• aka pre-experimental

static-group comparison design

X O (Group A) / O (Group B) - 2 existing groups compared, no pretest, no randomization

• cannot determine if groups were equivalent before the intervention

• sometimes used in pilot/feasibility studies but conclusion must be very cautious

what is internal validity

how well a study demonstrates that changes in DV are caused by IV, not other uncontrollable factors

threats to internal validity in experimental research (8)

history, maturation, testing, instrumental, selection bias, mortality/attrition, statistical regression, diffusion of treatment

history

an event outside the study occurs between pretest and posttest, affecting the DV (e.g., a hospital policy change)

**threat to internal validity

testing

taking a pretest influences performance on the posttest (practice effect)

**threat to internal validity

maturation

natural development, biological, or psychological changes occurs in subjects over time, independent of the IV

**threat to internal validity

instrumentation

changes in the calibration of instruments, or in observers/raters, over the course of the study

**threat to internal validity

selection bias

groups differ systematically at baseline d/t non-random assignment (common in quasi-experimental designs)

**threat to internal validity

statistical regression

subjects with extreme scores tend to score closer to the mean on retesting, regardless of intervention

**threat to internal validity

mortality (attrition)

subjects drop out of the study differentially across groups, biasing the remaining sample

**threat to internal validity

diffusion of treatment

control group subjects receive elements of the intervention (contamination), masking the true effect

**threat to internal validity

what is external validity

the extent to which findings can be generalized to other settings, populations, and times

** generalizability; can it be applied to other settings/groups?

threats to external validity (4)

selection effects, reactivity/Hawthorn effect, experimenter effects, novelty effect

selection effects

sample characteristics interact with treatment in ways that limit generalizability to other groups

**threat to external validity

reactivity (Hawthorne effect)

subjects behave differently simply because they know they are being observed/studied

**threat to external validity

experimenter effects

researcher’s expectations or behaviors unintentionally influence subject responses

**threat to external validity

novelty effect

subjects respond to newness of an intervention rather than its actual content

**threat to external validity

true experimental vs quasi-experimental designs:

randomization, manipulation of IV, control group, internal validity (chart)

randomization: always present in true, absent in quasi

manipulation of IV: present in both

control group: present in true, may be present in quasi

internal validity: strong in true, moderate in quasi

level of evidence: II for true, III for quasi

level of evidence of true experimental

Level II

level of evidence of quasi-experimental

Level III

what suggests a practice change? consideration?

a body of RCT evidence supports a practice CHANGE

a single quasi-experimental study suggest a practice CONSIDERATION

which 3 features must all be present for a design to be considered a true experiment

C. randomization, control, and manipulation of the IV

the symbol ‘X’ represents

D. the intervention or treatment

what is the Solomon four-group design is specifically used for

B. testing effects

what is this best classified as

A. a nonequivalent control group quasi-experimental design

rationale: no randomization → quasi-experimental

why is a washout period included in a crossover design

A. allow effects of the first treatment to dissipate before the next treatment begins

which of the following are threats to internal validity (SATA)

A. history

C. maturation

D. selection bias

what is this an example of?

C. statistical regression

the Hawthorne effect is a threat to which type of validity?

C. external validity

which of the following are true about quasi-experimental designs? (SATA)

A. they involve manipulation of the IV

C. they are commonly used in nursing research due to ethical and practical constraints

D. they generally provide Level III evidence

what does a one-group pretest-posttest design (O1 X O2) lack that makes it considered methodologically weak?

D. a control/comparison group