Appraising Quantitative Research Designs

Causality

is the study looking to show cause and effect?

needs an experimental design

counterfactual and effect

three criteria → temporal, relationship, and confounder

biological plausibility

Counterfactual

what would have happened to the same people exposed to a cause if they simultaneously were not exposed to the cause

Effect

represents the difference between what actually did happen when exposed to the cause and what would happen with the counterfactual condition

Temporal

the cause must precede the effect in time

Relationship

there must be a demonstrated association between the cause and the effect

Cofounder

the relationship between the presumed cause and effect cannot be explained by a third variable or cofounder, another factor related to both the presumed cause and effect cannot be the real cause

Biological Plausibility

the casual relationship should be consistent with evidence from basic physiologic studies

Research Designs

different designs are appropriate for different questions

must have the right design for each research question

experimental designs (RCTs) offer the strongest evidence of whether a cause (intervention) results in an effect (desired outcome)

RCTs are not appropriate for all research questions

Experimental Design Characteristics

intervention → manipulate IV and examine effect on DV

offer a good amount of control → control and experimental groups and measure to control extraneous variables

participants are randomly assigned to groups → make groups equal in regards to all factors besides type of intervention

Posttest Only Design

outcome data collected only after the intervention

symbolic representation

R = randomization

X = intervention

O = observation

Pretest Postest Design

outcome data collected both at baseline and after the intervention

symbolic representation

Crossover Design

subjects are exposed to 2+ conditions in random order

subjects serve as their own control

symbolic representation

Group Control Conditions

no intervention is used, control group gets no treatment at all

usual care or standard or normal procedures is used to treat patients

an alternative intervention is used

a placebo or pseudointervention presumed to have no therapeutic value is used

attention (extra attention but no intervention) control condition and delayed (intervention at a later date) treatment

symbolic representation

Experimental Designs Pros Cons

pros → most powerful for detecting cause and effect relationships

cons → often not feasible or ethical, Hawthorne effect (knowledge of being in a study may cause people to change their behavior), often expensive

Quasi Experimental Designs

involve an intervention but lack either randomization or control group

two main categories → nonequivalent or within subjects designs

Nonequivalent Control Designs

those getting the intervention are compared with a nonrandomized comparison group

Within Subject Groups

one group is studied before and after the intervention

Quasi Designs Pros Cons

pros → may be easier and more practical than true experiments

cons → they make it more difficult to infer causality, usually there are several alternative rival hypothesis for results

Nonexperimental Designs

when researchers do not intervene by controlling independent variables

not all independent variables of interest to nurse researchers can be experimentally manipulated → gender can’t be and smoking can’t ethically be

two types → correlational or description

Correlational Design

cause probing questions for which manipulation is not possible are typically addressed with this

correlational is an association between variables and can be detected through statistical analysis

weaker than RCTs for cause probing questions but different designs offer varying degrees of supportive evidence

Prospective Correlational Design

potential cause in the present is linked to a hypothesized later outcome

called a cohort study be medical researchers

stronger than retrospective designs in supporting causal inferences → but neither is as strong as experimental designs

Retrospective Designs

outcome in the present is linked to a hypothesized cause occurring in the past

one can be a case control design in which cases are compared to controls on prior potential causes

Descriptive Research

not all research is cause probing

the purpose of this study is to observe, describe, and document aspects of a situation

other research is correlational → the purpose is to describe whether variables are related, without ascribing a cause and effect connection

Nonexperimental Designs Pros Cons

pros → efficient way to collect large amounts of data when intervention or randomized is not possible

cons → does not yield persuasive evidence for casual inferences, not a problem when the aim is description but correlational studies are often undertaken to discover causes

Time Dimensions

cross correlational → data collected at a single point in time

longitudinal → data collected two or more times over an extended period, better at showing patterns of change, loss of participation occurs

follow up studies

Control

controlling external factors → achieving constancy of conditions, control over environment/setting/time, control over intervention via formal protocol

controlling participant factors → randomization, subjects as controls (crossover), homogeneity (restricting sample), matching, stats control (covariance analysis)

High Quality Research

statistical conclusion validity → the ability to detect true relationships statistically

internal validity → extent to which it can be inferred that the IV caused the DV

external validity → generalizability of observed relationships across samples, settings, or time

construct validity → degree to which key constructs are adequately captured

Conclusion Validity Threats

low statistical power (sample too small)

weakly defined cause → IV not powerful

unreliable implementation of treatment → low intervention fidelity

Internal Validity Threats

temporal ambiguity

selection threat → biases arising from preexisting differences between groups being compared (non experimental threat)

history threat → other events co-occuring with casual factors that can also affect outcomes

maturation threat → processes that result simply from passage of time

mortality/attrition threat → differential loss of participants from different groups (experimental threat)

External Validity Threats

inadequate sampling of study participants

enhancing internal validity can sometimes have adverse effects on external validity