quantitative research designs

experimental designs types

- qualitative (descriptive) --> describe populations

- observational (exploratory) --> find relationships

- experimental (explanatory) --> cause and effect

experimental designs

- pretest-posttest

- posttest-only

- factorial designs

- repeated measure designs

selecting a design

- how many IVs are being treated?

- how many levels does each independent variable have, and are these levels experimental or control conditions?

- how many groups of subjects are being tested?

- how will subjects be assigned to groups?

- how often will observations of response be made?

- what is the temporal sequence of interventions and measurements?

pretest-posttest control group design

- basic RCT (one IV)

- compare 2 or more groups

- parallel groups: one group receives intervention, control group: sham, placebo, usual care, or another treatment

random assignment: exerimental group --> measurement : ____ --> IV: ____ --> measurement

pretest; experimental intervention; posttest

random assignment: control group --> measurement : ____ --> IV: ____ --> measurement

pretest; control condition; posttest

pretest-posttest control groups:

- placebo/sham

- treatment control

- multi group design

multi group design

- ability to compare several treatment and control conditions

- can accommodate any number of levels of one IV

pretest-posttest design validity

- strong internal validity

- pre-test establishes initial equivalence of groups

- strengthens evidence that effects of the treatment account for the group differences in posttest scores

analysis: pretest-posttest

often analyzed based on the change from pretest to posttest using difference scores

pretest-posttest types

- interval and ratio

- ordinal

interval and ratio tests for pretest-posttest

- independent t-test of 2 groups

- one wat anova for 3 or more groups

ordinal tests for pretest-posttest

- mann-whitney U : 2 groups

- kruskal-wallis: 2 or more groups

posttest-only control group design

- identical to pretest-posttest design EXCEPT no pretest for either group

- parallel groups: one group receives intervention. control group= sham, placebo, usual care, or another treatment

why a posttest only

- if the pretest is impractical or potentially reactive

- decreases bias if pretest could influence posttest scores

- increasing external validity by avoiding this form of bias

design validity

because there is no pretest to demonstrate group equivalence with randomization, this design is best done with large same size to increase probability of balancing out interpersonal characteristics

factorial designs for independent groups

- allow for more complex and often clinically realistic phenomenon

- used to account for the interaction of several variables

- 2 or more IVs

- random assignment to various combinations of levels of variables

- usually involve 2 or 3 variables

dimensions

- # of factors or IVs

- two way/factor = 2 IVs

- 3 war/factor = 3 IVs

3x3 - how many factors? levels? groups?

- 3 factors

- 3 levels

- 9 groups

2x3x4 - how many factors? levels? groups?

- 3 factors

- levels: 1:2, 2:3, 3:4

- 24 groups

two way factorial design

- 2 independent variables (joint protection, hand exercise)

- each IV has 2 levels (2x2)

- 4 total groups

2 way factorial design allows us to ask 3 questions of the data

1. is there a differential effect of joint protection training vs information leaflet?

2. is there a differential effect of hand exercises vs information leaflet

3. what is the interaction between joint protection training and exercise?

- numbers 1 and 2 are answered using MAIN EFFECT of each IV

- number 3 is answered using the INTERACTION EFFECT between the 2 IVs

analysis: factorial designs - ANOVA

- 2 way and 3 way anova are most commonly used to examine the main and interaction effects

using ANOVA we can answer the following questions

- what is the effect of modality independent of medication?

- what is the effect of medication, independent of modality?

- what is the combined effect or interaction of modality and medication?

repeated measures design

- one group is tested under all conditions

- the subject acts as their own control

- advantage: control for individual differences

- one way repeated measures

- two way design with 2 repeated measures

- mixed design

repeated measures design cont.

- simplest form of repeated measures

- randomization in order of application of repeated conditions

- analyzed with one-way repeated measures ANOVA

things to consider: repeated measures designs

- practice effects

- carryover effects

- order effects

practice effects

- learning effect from repetition of task over and over

- can randomize order to account for this

carry over effects

- influence of prior treatment on outcomes; exposure to multiple conditions

- reduced by allowing for time between conditions

order effects

- biasing effect of the test order

- one solution is to randomize order of presentation for each subject, latin square can be used

latin square

randomization to determine which group gets which sequence

carryover designs

- 2 levels of IVs are repeated

- control for order effects: counterbalancing the treatment conditions

- half receive intervention A followed by intervention B, the other half receives intervention B followed by intervention A

- should only be used if the patient's condition/disease will not change much over time

- when the treatment has cumulative effects, a WASHOUT PERIOD is needed

analysis: crossover design

researchers typically group scores by treatment condition regardless of the order they were given

2 types of cross over design

- interval and ratio

- ordinal

interval and ratio

- paired t-test to compare change in scores

- two-way ANOVA with 2 repeated measures to compare pretest and posttest across treatment conditions

ordinal

wilcoxon signed ranks test to compare change in scores

two way design with two repeated measures

- 2 independent variables

- each subject exposed to all conditions and time all time measures

- when subject responses are compared over successive time periods, time becomes an independent variable

- example: studying validity of HR monitors

- 3 devices worn, HR measurements taken at 4 time points

- 3x4 design: type of wrist device (3 levels) and measures over time (4 levels)

- analysis: 2 way ANOVA with 2 repeated measures to analyze differences across main effects and interaction effects

mixed design

-2 independent variables, one repeated across all subjects, other randomized to independent groups

mixed design example

- women with pelvic pain randomly assigned to 2 treatment groups (PT or PT w/focus on stabilizing exercises). assessments by blinded assessor at baseline, post intervention, and 1 year postpartum

- exercise programs were randomly assigned: independent factor

- time: repeated factor

- 2 way design with one repeated measure, or a 2 x 3 mixed design

mixed design analysis

2 way ANOVA with one repeated measure

- main effects and interaction effects

quasi-experimental - similar to experimental but LACK:

- random assignment

- comparison groups

- both

quasi-experimental

- used when randomization and/or control groups are not possible or unethical

- threats to internal validity

- time series designs, non-equivalent group designs

one group pretest-posttest

- one set of repeated measures taken before and after on one group of subjects

- IV is time with 2 levels (pre and post)

- no comparison group - validity concern

- previous research can be used to document behavior of a control group

analysis of one group pretest-posttest

- paired t-test

- non parametric: wilcoxon signed ranks test

repeated measures design of quasi experimental

- one set of repeated measures taken before and after on one group of subjects

- IV is time

- multiple pre/post tests can allow trends to be observed

- no comparison group - validity concern

- analysis: one-way repeated measures ANOVA

interrupted time series design

- series of measurements that are interrupted by one or more treatment occasions

- IV is time

- one group studied

- most effective when serial data can be collected at evenly distributed intervals

interrupted time series benefit is to

show trends over extended periods

in which of these patterns would you be justified in assuming the intervention has an effect?

non-equivalent groups

- similar to a pretest-posttest design EXCEPT subjects are not randomly assigned

- may already be a part of an intact group (class or personal circumstances)

- subjects may self-select their group preference limitation and potential bias due to lack of random assignment

analysis: non-equivalent pretest-posttest

- interval and ratio

- ordinal

tests such as the t-test, analysis of variance, and chi square are often used

to test for differences in baseline measures

regression analysis or discriminant analysis may be the most

applicable approach to determine how the dependent variable differentiates the treatment groups

single subject design

- serial observations of individuals before, during, and after interventions considering clinical outcomes

- patient level investigations

- provide input on EBP

- improve decision making in the clinical setting

- real time data

single subject design also called

- single case design designs

- N of 1 trials

- time series designs

- small N designs

focus on the individual

- individual response/results hidden by conclusions drawn from the average

limitations of RCTs

- can a treatment effect be generalized to all people?

- RCTs may not be adequate for clinical decision-making for an individual patient

individualized care

- no one size fits all

- patient participation in decision-making

research question

- comparisons among several treatments, components of treatments, or treatment vs no treatment

- may have a directional or non-directional hypothesis

single-subject design structure

- measuring the response of the target behavior at frequent and regular intervals

- at least 2 testing periods

(baseline A)- control condition, intervention (B), A-B design

SSD vs Case report

- case report provides description after the fact

- used to find patterns and generate hypotheses

- case reports do not involve manipulation of variables

single subject designs

- withdrawal designs (A-B-A and A-B-A-B)

- multiple baseline

- multiple and alternating treatment

- changing criterion (raising the bar)

- N-of-1 (randomized crossover)

withdrawal designs

- replicates one baseline phase following intervention

- controls internal validity as it is unlikely that confounding factors would occur before and after (giving credit to intervention)

- A-B-A-B (additional intervention phase offering even more evidence)

multiple baseline design

- 3 or more comparisons (or subjects), each of who is assigned a different baseline length

- used if withdrawal is not practical

- baseline data are available so that temporal effects do not contaminate results

- multiple baseline across settings: one individual, multiple settings, same treatment applied across 2 or more environmental conditions

alternating treatment designs

- rapid alternation of 2 or more interventions or treatment conditions each associated with a direct stimulus

- used to compare treatment with placebo or treatment vs treatment

multiple treatment

one treatment following baseline, the withdrawal of that treatment then intro of one or more additional treatments (A-B-A-C design)

changing criterion

- adjusting treatment over time as patient progresses

- increasing goals incrementally

N-of-1 Trials

- used as an active decision making tool

- which treatment should I use for this patient?

- most commonly used to determine individual effectiveness of drugs

crossover design of N-of-1 Trials

treatment and placebo are systemically and randomly altered until the patient and clinician reach a decision on a preference for one

N-of-1 Trials criteria

- there is uncertainty of the effectiveness of a treatment for an individual patient

- the condition is chronic or slowly progressing, with frequently occurring symptoms

- a washout period is safe and feasible

- the treatment effect has a rapid onset with no carryover

- the patient and clinician can be blinded to the intervention

- a clinically relevant outcome can be identified and measured, including symptoms or functional activities

- the patient is motivated to participate and is expected to be compliant

- resources are available to cover costs of intervention and time commitments

- the study is ethical

observational research types

- cohort studies

- case-control studies

- correlation and predictive research (diagnosis, prognosis)

exploring relationships - observational design

- comparing groups without assigning exposure

- descriptive: characterizing populations

- analytic: examining group differences to determine how exposure influences outcomes

longitudinal studies

- following subjects across time and collecting data at intervals

- prospective: direct recording or measurements

- retrospective: examining previously collected data

cohort studies

- prospective or retrospective

- types of cohorts: inception, birth, historical, development

- subject selection: free from target outcome at baseline, susceptible to outcome

- challenges: classifying exposure, attrition

case-control studies

- determining if the groups different in terms of exposure history or presence of risk factors

- selection of cases based on case definition

- selection of controls: restrictions applied to cases should carry over

- challenges: selection bias, observation bias, recall bias, confounding factors

- matching controls to minimize bias

cross-sectional studies

- taking a population "snapshot" at a single point in time

- collect data about exposure and outcome at the same time

- may be used because it is more efficient than longitudinal studies

- challenges include risk of reverse causation