Evidence Based

Understand research design and methodology terminology 

• Research design with the potential for determining causality can be broadly categorized into 2 types - observational and experimental

• How much control over subject exposure does researchers have?

  • In observational design, no control

    • The researcher merely observed the interplay of independent variable (drug exposure) with the dependent variable (outcome of interest)

    • Most commonly used is the randomized controlled trial (RCT)

  • In experimental design, some or total control 

    • The researcher controls the independent variable (treatment) that is likely to have an impact on the dependent variable (outcome)

    • Most commonly used include

      • Cohort 

      • Case control 

      • Cross-sectional 

      • Case series

      • Case reports 

Describe the classification of research designs in clinical research 

Study Purpose

Descriptive studies: describes or summarize information about the diseases, events, or characteristics of study subjects without making any causal inferences 

• Often used to generate data for hypothesis 

• Who: refers to the demographic characteristics, such as age and gender of the study population. Usually involves a single individual or group with no comparator.

• What: defines a phenomenon based on specific inclusion and exclusion criteria

• Why: provides clues about the possible causal mechanism for further study

• When: the time period related to the occurrence of the phenomenon.

• Where: the place of occurrence

• So what: relates to information that could identify the importance of the issue to the community (public health)

Analytical studies: aimed at understanding the relationship and/or causal mechanism that may exist between 2 or more variables 

• They can be experimental or observational

• Complex and resources intensive 

• Usefulness of these studies lies in their ability to test the relationship and causal pathways 

Time orientation 

• Prospective: where the researcher collects the data after the study onset by following individuals over a period of time

  • Strength: define the research variables 

  • Limitation: resource (time and cost) intensive

  • All experimental designs and some observational designs are prospective designs 

• Retrospective: involves the evaluation of data for past events or existing data, such as medical records, to achieve the research objective 

  • Strength: less resource intensive using existing data or past events 

  • Limitation: no control over the variables used, how they are defined, or how collected 

Investigator orientation

• Experimental Studies: investigators assign participants to intervention/treatment groups based on randomization or other methods of selection 

  • Experimental studies have the provide strongest causality 

  • Examples: randomized controlled trials 

• Quasi-Experimental Studies: investigators assign participants to intervention/treatment groups - no randomization 

  • Provides weaker evidence for causality than experimental and more than observational studies

  • Examples: nonequivalent groups, pretest-posttest, interrupted time series

• Observational Studies: investigators do not assign participants to groups and rely on pre-existing differences. The investigators observe the natural course of events without influence.

  • They have the lowest casual strength but strong external validity

  • Ex: case control studies, cohort studies 

Experimental setting 

• Research clinical trial setting 

• Everywhere else

Discuss common research methodologies in clinical research: RCT

 Randomized Controlled Trials (RCTs)

• First element involved the randomization of study participants to:

  • Intervention: the group in which participants are provided an intervention (also called “experimental” or “treatment” group)

  • Control: the group that is provided conventional or no intervention 

• The second essential elements of RCTs - always prospective;

  • Patients in the study group are followed after the intervention to evaluate changes in the clinical outcome 

• Randomization increases internal validity of RCTs

  • Distribution of all observed as well as unobserved baselines reduces any systematic differences among participants in influencing study results 

• Inclusion criteria used with RCTs increase the internal validity of study results also contribute toward restricting their external validity (generalizability)

Methods of Reducing Bias

Allocation concealment

• What:

  • Preventing investigators from knowing which group a subject will be allocated to at the time of recruitment/enrollment in trial

  • Accomplished with central randomization or sealed envelopes

• Why: 

  • Reduces potential for selection bias in the study sample

    • Knowing which group a subject will be assigned to may lead an investigator to choose not to enroll a subject

Blinding

• What: 

  • Preventing patients, investigators, and/or data analysts from being aware of treatment received during study and analysis of results

    • Patients/investigators: use placebos and/or sham procedures to blind

    • Data analysts: use coded data for analysis to mask treatment group

• Why: 

  • Prevent knowledge of treatment assignment from influencing use of other interventions, reporting of results, or analysis choices

• Who:

  • Open-label: NOBODY is blinded; treatment/group assignment is known to participants and investigators

  • Single-blind: EITHER participants OR investigators are unaware of treatment/group assignment

  • Double-blind: BOTH participants AND investigators are unaware of treatment/group assignments

  • Triple-blind: EVERYONE is blinded, including participants, investigators, data analysts 

Use of dummies (placebo)

• What: A form of placebo used to disguise treatments/interventions that are physically dissimilar

  • “Double-dummy” is common, with two matching placebos, one for each intervention

    • Ex: Inhaled insulin vs. injectable insulin 

95% Confidence Interval & Dot and Whiskers

Types of RCTs:

• SUPERIORITY study: investigators attempt to show an intervention is better than a comparator (control)

  • Traditional comparative trial used in most placebo-controlled RCTs

• EQUIVALENCY study: investigators attempt to show an intervention is as good as a comparator (control)

  • Useful for bioequivalence studies for approval of generic drugs

• NON-INFERIORITY study: investigators attempt to show an intervention is not worse than a comparator (control)

  • Useful when comparing a new agent to an existing gold standard

RCT: advantages and disadvantages

Observational Design 

• Researchers observe the relationship between the study variables (independent and dependent) in a natural setting

  • There is no randomization of participants into experimental and control groups. 

  • The key element in observational studies is nonrandomization of the independent variable, which is an exposure of interest like medication use or an intervention. 

• In observational studies, investigators collect data regarding exposure and outcomes using 

  • primary data techniques like interviews and surveys 

  • Secondary data collected previously for other purposes like medical charts.

Common observational studies include: case reports, case series, cross-sectional studies, case control studies, and cohort studies.

Case series/reports

• Case report: involves a study of a single case of a new disease or manifestation 

• Case series: involves study of multiple similar cases 

  • Brings attention to unusual clinical situations that otherwise may have been missed, like clinical insight into rare events and AD/beneficial drug effects

  • Case Report/Series are descriptive and considered to be at bottom of research hierarchy 

Cross Section Study

Cross section studies: measure the exposure and outcome of interest at the same point in time

• Provide a snapshot of the presence of outcome and/or exposure status in the population 

• Used to determine prevalence or the proportion of individuals with a disease or outcome of interest at a given point in time 

Case control study

Case control study involves comparison of exposure status among individuals with the disease or outcome of interest (cases) and those without the disease or outcome (controls)

• Cases + controls are both identified from the same source population, with the only difference being that the former experienced the outcome of interest while the latter did not 

• Retrospective in nature

• Case control studies are the design of choice to study a rare outcome and in situations where there is a long latency period between exposure and the occurrence of the outcome  

• Risk for bias if flaws in method used to identify the control group or determine exposure status 

Cohort Study

Cohort Study: observational studies wherein two groups, the exposed and the unexposed, are followed (prospectively or retrospectively) over a period of time until the development of outcome of interest 

• At baseline, none of individuals in the two groups have the outcome

• Groups r defined based on the exposure status (exposed vs. unexposed) and observed for a given time period going forward 

• Frequency of occurrence of outcome among the exposed group is compared to the exposed group 

• Exposure proceeds outcome - cohort are most powerful observational designs

• Divided into 2 types

  • Prospective cohort study: exposed and unexposed groups r classified at baseline and these groups are then followed in future to determine the occurrence of outcome of interest in the two groups 

  • Retrospective cohort study: researcher uses previously collected (historical) data to identify exposure status and occurrence of outcome in the study group

• Strength: ability to ascertain temporality when examining the relationship between exposure and outcome 

Meta-analysis (Systematic Review)

Importance (why do them?)

• To increase statistical power for primary endpoints and for subgroups

  • Particularly for smaller studies with negative results

• To resolve uncertainty when studies disagree

  • Conflicting evidence with previous published studies

• To improve the estimates of size/magnitude of effect

• To answer new questions not posed at the start of individual trials

• To bring about improvements in the quality of the primary research (ie. help to generate new hypotheses to test in RCTs)

How similar (homogenous) are pooled studies?

• Heterogeneity refers to the variability or differences among data points, samples, populations, or study results, indicating that they are not all the same or similar. In a meta-analysis, the more heterogeneity between the studies the weaker the results of the pooled analysis.

• Tests for heterogeneity in meta-analyses include:

  • Cochran’s Q test 

    • Based on a chi-square distribution, it generates a probability that, when large, indicates larger variation across studies rather than within subjects within a study (detects if heterogeneity present)

    • The underlying null hypothesis assumes that the true treatment effect is the same across studies and variations are simply caused by chance

    • Limitation of Cochran’s Q test: it might be underpowered when few studies have been included or when event rates are low. Therefore, it is often recommended to adopt a higher P-value (rather than 0.05)

  • The I2 index

    • I2 provides an estimate of the percentage of variability in results across studies that is due to real differences and not due to chance (the amount of heterogeneity that is present – generally, 0.25 is low, 0.5 is moderate, 0.75 or greater is high)

    • The limitation of I2 is that it provides only a measure of global heterogeneity but no information for the factor causing heterogeneity