Chapter 6: Choosing Interventions for Practice: Designs to Answer Efficacy Questions

randomization

R

nonrandomization

N

treatment

X

dependent variable or outcome

O

between-group comparison

comparison is made to identify the differences between two or more groups with different individuals in the groups

within-group comparison

comparison with the same individuals in the conditions being compared, most often a comparison of scores before and after an intervention

interaction effect

combines the between- and within-group comparisons, so an interaction is said to occur when there is a difference in how a group performs over time

analyses

Different types of _____________ used to determine if two groups differ over time.

- One common approach involves treating the pretest as a covariate and analyzing the difference in the posttest scores of the two groups using an analysis of covariance (ANCOVA).

main effects and interactions

any time the lines cross there is potential for an interaction

if the p value is lower that means the probability was due to chance.

designs without control groups

Technically not an experimental design

Pre-experimental design – may be used to determine whether an intervention has the potential to make a difference, before investing the time and money in a more extensive RCT

Level IV in hierarchy of evidence

Absence of control group = big limitation

O X O

randomized control trials (RCTs)

Extremely valuable to intervention research

Level II evidence

Control group - does not receive intervention or may receive placebo or standard treatment

Maturation may pose a threat to internal validity

Some RCTs may only have posttest

- Typically when a pretest would influence the outcome of the posttest (a testing threat to validity) or those in which it is expected that all participants will start out at a similar point

true RCT

R O O

R O X O (control group)

WITH A PRETEST

with comparison to standard treatment RCT

R O Xa O

R O Xb O

posttest only RCT

R X O

R O (control group)

crossover design

Level II evidence if participants are randomly assigned to groups

All participants receive the same treatment, but in a different order

Treatment vs. no treatment may be compared

- R O X O O

- R O O X O

Two interventions may be compared

- R O Xa O Xb O

- R O Xb O Xa O

nonrandomized controlled trials (NRCTs)

Participants do not have an equal chance of being assigned to a condition

Can lead to bias or differences between groups

Level III evidence

May ask for volunteers for intervention group and then match controls

May be used for pragmatic or ethical reasons

Cluster randomized control trial

Also referred to as quasi-experimental studies or nonequivalent control group designs

cluster randomized control trial

settings are randomly assigned to group

notation with true control group (NRCT)

N O O

N O X O

notation with comparison treatment (NRCT)

N O Xa O

N O Xb O

factorial designs

Can be randomized or nonrandomized

Distinguished from other designs in that they have more than one independent variable

Additional independent variable typically included to determine if the intervention had differential effects on that additional variable

Described in terms of the number of levels within each independent variable

For example, a study with two levels of the intervention, A and B, and two levels of gender, male and female, would be a 2 x 2 design

Results include main effects and interaction

single-subject designs

Compare an individual's response under different conditions

Cause-and-effect relationships can be inferred from a strong single-subject design study when there is a clear difference between behavior that occurs when an intervention is present and that which occurs when the intervention is absent.

Within-subjects design

Different notation

ABA design

Single-subject design

Example:

- The first A represents the initial phase of the study and comprises a baseline observation and collection of data.

- Then an intervention is provided, which is represented by the B phase. The same observation and collection of data continue.

- Finally, the intervention is removed, and observation and data collection continues, indicated by the second A.

- If a cause-and-effect relationship exists, a change in behavior occurs during the intervention

generalizability

major limitation of single-subject designs

replication

important in single subject designs

- with multiple participants, if the results are similar for all participants, you can have greater confidence that the intervention caused the outcome.

multiple baselines

use of __________ __________ strengths in single-subject designs

- ABAB design or ABABAB design

- If improvement during the intervention and a decline at baseline are consistently shown, evidence-based practitioners can be more assured that the intervention was effective and caused the change in behavior.

retrospective intervention studies

Researcher looks back at something that has already occurred and uses existing records to collect the data

Not experimental, observational

Sometimes these studies are called retrospective cohort studies because they utilize and compare existing groups (cohorts) of individuals.

disadvantage of RI studies

conditions cannot be controlled, and numerous threats to internal validity are likely present in the existing conditions.

advantage of RI studies

examine practices that have taken place in real-world clinical situations, the study can have greater external validity

sample size and intervention research

Larger sample reduces the likelihood of making a Type II error

Reduces sampling error so that the results of the study are more likely to reflect the true population

Larger number of participants = more reliable data

PEDro Scale

Numerical rating applied to a study to objectively assess the methodological quality of an individual study