Quantitative Research Design

Key features of quantitative research design

intervention, comparisons, control over confounding variables, blinding, time frames, relative timing, location

Intervention design options

experimental (RCT), quasi-experimental, nonexperimental/observational design

Comparisons design options

same participants at different times or conditions OR different participants

Control over confounding variables design options

randomization, crossover, homogeneity, matching, statistical control

Blinding design options

blinding of participants, interventionists, other staff, data collectors

Time frames design options

cross-sectional, longitudinal design

Relative timing design options

retrospective (case control), prospective (cohort) design

Location design options

setting selection; single site versus multisite

Causality

many (if not most) quantitative research questions are about causes and effects; research questions that seek to illuminate causal relationships need to be addressed with appropriate designs

Counterfactual model of causality

what would have happened to the same people exposed to a “cause” if they simultaneously were not exposed to the cause

Effect

represents the difference between what actually did happen when exposed to the cause and what would happen with the counterfactual condition

Criteria for causality

temporal, relationship, confounder

Temporal

the cause must precede the effect in time

Relationship

there must be a demonstrated association between the cause and the effect

Confounder

the relationship between the presumed cause and effect cannot be explained by a third variable or confounder; another factor related to both the presumed cause and effect cannot be the “real” cause

Biological plausibility

the causal relationship should be consistent with evidence from basic physiologic studies

Experimental designs (RCTs)

offer the strongest evidence of whether a cause (an intervention) results in an effect (a desired outcome)-- that's why they are high on evidence hierarchies for questions about causes and effects

Therapy

RCT/experimental design --> quasi-experimental --> cohort study --> case control --> descriptive/correlational

Prognosis

cohort study --> case control --> descriptive/correlational

Etiology/harm

RCT/experimental design --> quasi-experimental --> cohort study --> case control --> descriptive/correlational

Within/between group comparison

can use either or both in a study

Within-groups comparison

comparisons about measurements made with same subjects at different points in time (ex. 2 observations at different times but done on same group after an intervention)

Between-groups comparison

comparisons made with more than one group of subjects at one or more points in time (ex. control and experimental groups, experimental gets intervention, 2 separate comparisons made before and after, and control gets evaluated, too)

Intervention

The researcher does something to some subjects—introduces an intervention (or treatment); pre-and posttests

Control

the researcher introduces controls, including the use of a control and experimental groups

Randomization

the experimenter assigns participants to a control or experimental condition on a random basis in order to make the groups equal with regard to all other factors except receipt of the intervention

R means

randomization

O means

measurement at a point in time

(observation, data collection)

X means

intervention or treatment (can sometimes be listed as T)

Posttest-only (or after-only) design

outcome data collected only after the intervention, symbolic representation (R x O; R ... O)

R in posttest-only design

randomization

X in posttest-only design

receipt of intervention

O in posttest-only design

observation/measurement of dependent variable

Pretest-posttest (before-after) design

outcome data collected both at baseline and after the intervention, symbolic representation (R ... O x O; R ... O ..... O)

Crossover design

subjects are exposed to 2+ conditions in random order, subjects serve as their own control

Symbolic representation for crossover design

R O Xa O Xb O

R O Xb O Xa O

Experimental condition-proxy

must be designed with sufficient intensity and duration that effects might reasonably be expected, researchers describe the intervention in formal protocols that stipulate exactly what the treatment is

Intervention fidelity

whether the treatment as planned was actually delivered and received

Control group conditions (counterfactuals)

no intervention is used; control group gets no treatment at all, “usual care” or standard or normal procedures is used to treat patients, an alternative intervention is used (e.g., auditory vs. visual stimulation), a placebo or pseudointervention, presumed to have no therapeutic value, is used, attention control condition and delayed treatment (wait-listed)

Attention control

extra attention but not the active ingredient of the intervention

Delayed treatment ("wait-listed controls")

the intervention is given at a later date

Control group conditions

attention control, delayed treatment

Symbolic representation for control group conditions

R O X O ..... O

R O ... O X O

Advantages of experiments

most powerful for detecting cause and effect relationships

Disadvantages of experiments

often not feasible or ethical, Hawthorne effect (knowledge of being in a study may cause people to change their behavior), often expensive

Quasi-experiments

involve an intervention but lack either randomization or control group

Two main categories of quasi-experimental designs

nonequivalent control group designs and within-subjects designs

Nonequivalent control group designs

those getting the intervention are compared with a nonrandomized comparison group, if pre intervention data are gathered, then the comparability of the experimental and comparison groups at the start of the study can be examined-- nonequivalent control group pretest–posttest design

Within-subjects designs

one group is studied before and after the intervention

Symbolic representation for nonequivalent control group designs

O1 x O2

O1 x O2

Nonequivalent control group designs #2

without pre intervention data, it is risky to assume the groups were similar at the outset-- nonequivalent control group posttest only is much weaker

Symbolic representation for nonequivalent control group designs #2

X O1

.... O1

Within-subjects quasi-experiments

one-group pretest–posttest designs typically yield extremely weak evidence of causal relationships; symbolic representation = O1 x O2

Time-series designs

within-subjects quasi-experiments that gather preintervention and postintervention data over a longer period; symbolic representation = O1 O2 O3 O4 x O5 O6 O7 O8

Advantages of quasi-experiments

may be easier and more practical than true experiments

Disadvantages of quasi-experiments

they make it more difficult to infer causality, usually there are several alternative rival hypotheses for results

Nonexperimental studies

if researchers do not intervene by controlling independent variable

T/F-- All independent variables (“causes”) of interest to nurse researchers can be experimentally manipulated

false

Correlational designs

cause-probing questions (e.g., Prognosis or Etiology/harm questions) for which manipulation is not possible are typically addressed with a correlational design

Correlation

an association between variables and can be detected through statistical analysis

Correlational studies

weaker than RCTs for cause-probing questions, but different designs offer varying degrees of supportive evidence

Prospective correlational design

a potential cause in the present (e.g., experiencing vs. not experiencing a miscarriage) is linked to a hypothesized later outcome (e.g., depression 6 months later)

Cohort study

name of prospective correlational design by medical researchers

Prospective designs

stronger than retrospective designs in supporting causal inferences—but neither is as strong as experimental designs

Retrospective correlational design

an outcome in the present (e.g., depression) is linked to a hypothesized cause occurring in the past (e.g., having had a miscarriage)

Case-control design

one retrospective design in which "cases" (e.g., those with lung cancer) are compared to "controls" (e.g., those without lung cancer) on prior potential causes (e.g., smoking habits)

Purpose of descriptive research

to observe, describe, and document aspects of a situation

Descriptive research example

ascertaining the prevalence of a health problem

Descriptive correlational research

the purpose is to describe whether variables are related, without ascribing a cause-and-effect connection

Disadvantage of nonexperimental research

does not yield persuasive evidence for causal inferences

Advantage of experimental research

efficient way to collect large amounts of data when intervention and/or randomization is not possible

Cross-sectional design

data are collected at a single point in time

Longitudinal design

data are collected two or more times over an extended period, includes follow-up studies, and is good at showing patterns of change and at clarifying whether a cause occurred before an effect (outcome)

Challenge in longitudinal studies

attrition or the loss of participants over time

Controlling external factors (such as research context)

achieving constancy of conditions, control over environment, setting, time, control over intervention via a formal protocol: intervention fidelity

Controlling participant factors

randomization (subjects as own controls (crossover design)), homogeneity (restricting sample), matching, statistical control (e.g., analysis of covariance)

Characteristics of good quantitative research design

statistical conclusion validity, internal validity, external validity, construct validity

Statistical conclusion validity

the ability to detect true relationships statistically

Internal validity

the extent to which it can be inferred that the independent variable caused or influenced the dependent variable

External validity

the generalizability of the observed relationships across samples, settings, or time

Construct validity

the degree to which key constructs are adequately captured in the study

Threats to statistical conclusion validity

low statistical power (e.g., sample too small), weakly defined "cause"—independent variable not powerful, unreliable implementation of a treatment—low intervention fidelity

Threats to internal validity

temporal ambiguity, selection threat, history threat, maturation threat, maturation/attrition threat

Selection threat

biases arising from preexisting differences between groups being compared-- this is the single biggest threat to studies that do not use an experimental design; tendency of subjects to have similar characteristics-- moti vation, education

History threat

other events co-occurring with causal factor that could also affect outcomes; influence of events on DV while study is being conducted, not part of the study

Maturation threat

processes that result simply from the passage of time; changes within study subjects over time

Mortality/attrition threat

differential loss of participants from different groups-- typically a threat in experimental studies; subjects who withdraw from study prior to completion, attrition rate = dropout rate, <20% is goal

History threat reduction

random selection or assignment

Maturation threat reduction

control group, random assignment, baseline data

Testing for threat to internal validity

multiple testing might influence subjects response on subsequent testing, threat reduction, control group tested same # times or prolong length of time between tests

Threat reduction for threat to internal validity

post test only design

Threat reduction for selection threat

random sampling, random assignment

Threat reduction for mortality threats

collection of demographic variables from all subjects for future comparison those who compete, communicate clear instructions and expectations regarding participation in the study

Instrumentation threat

inconsistency in data collection

Threat reduction for instrumentation threat

comprehensive training of data collectors, reliability and validity of instruments

Threat to external validity

person, time, place; inadequate sampling of study participants, unfortunately, enhancing internal validity can sometimes have adverse effects on external validity

Threats to construct validity

is the intervention a good representation of the underlying construct?, is it the intervention or awareness of the intervention that resulted in benefits?, does the dependent variable really measure the intended constructs?

T/F-- An experimental research design involves a nonrandomized controlled trial

false

Which characteristic is a key criterion for causality?

Cause occurring before the effect

3 multiple choice options

T/F-- A true experiment requires that the researcher manipulate the independent variable by administering an experimental treatment (or intervention) to some subjects while withholding it from others

true