Clinical Investigations exam 2

What are observational studies?

Where the researcher is documenting a naturally occuring relationship between the exposure and the outcome being studied

Are observational studies descriptive or analytical?

Descriptive

how does descriptive defer from analytical?

Lowest level in evidence hierarchy

Does not formally try to answer a research question or establish a relationship between variables

Types of descriptive studies?

Case reports

Case series

Descriptive surveys

What is the first step to discover?

Observation

Observational study designs

Cross-sectional study

Case-control study

Cohort study

What is a cross-sectional study?

A prevalence study (risk)

Basic descriptive study

Examines relationship between outcome or exposure and other characteristics in a population at one point in time

gather data on both exposure and outcome at the same time

cannot measure incidence of outcome

What is a case-control study?

Compares those with outcome vs those without (control)

looks for potential causes or exposures

start with outcomes (case and control groups) and work backwards to look for exposures

What is a cohort study?

Incidence study

Measures rate of outcomes, starts with population without outcome

Follows development of outcome in that population over time

can be retrospective or prospective, but both are longitudinal

What is a signficant limitation of observational studies?

cannot show causality, only associations

When are observational studies often done?

When RCTs cant be done

Advantages of observational studies

Results can provide preliminary data to inform design and conduct

may have more generalizability than an RCT (depends)

Prevalence vs incidence

Prevalence: Number of people with outcome at a given time

Incidence: Number of new cases over a period of time

What is the most feasible design when disease outcomes are rare?

Case-control studies

When is exposure data collected in case-control studies

Retrospectively

Pros of case-control study

Quick and cheap (fewer people needed than cross-sectional)

Only feasible method for rare diseases or long lag between exposure and outcome (because you start with outcome and work backwards)

Cons of case-control study

Reliance on recall or records to determine exposure status (recall bias)

confounders

selection of control groups is difficult

Potential for selection bias

Nested case-control

Conducted within a defined cohort that has been followed over time

Take a active cohort study and pull from it to get a case-control (exposure is better accounted for)

What is the strongest observational study design?

Cohort study

When is cohort study used?

When looking for a difference in the risk (incidence) of a disease over time

When is data collected for cohort study?

Collected prospectively (collect as it goes)

Examined retrospectively (examine data after study)

Prospective cohort study

Looks foward/to the future. Events haven’t occured yet

Strongest cohort design

Retrospective cohort study

Look back in time to study events that have already occurred

More possibilities of counfounding/bias (recall bias)

Key points for retrospective observational studies

Pros

• Inexpensive (data already exists)

• Shorter time needed

• Easier to get large number of subjects

• Useful for exposures that no longer occur

Cons

• Information/data may be less complete or inaccurate

• Potential recall bias

Key points for prospective observational studies

Pros

• Info/data may be more accurate/complete

• Direct access to subjects, more reliable recall

Cons

• Expensive

• Long time needed

• More difficult to recruit large number of subjects

• Exposure status and diagnostic methods may change

• Loss of subjects over time can be substantial and affect results

Pros and cons of cohort study

no allocation of exposure made by the researcher

Best for studying effects of risk factors on an observed outcome

Pros

• Ethically safe

• Participants can be matched

• Can establish timing and direcitonality of events

• Eligibility criteria and outcome assessments can be standardized

• Can study several outcomes at once

Cons

• Control/unexposed can be difficult to identify (retrospective)

• Exposure could have hidden confounder (retrospective)

• Blinding can’t be done

• Hard to do rare diseases (need large number/follow ups)

• Susceptible to bias (esp retrospective)

• Can be time-consuming and expensive

• Loss to follow up (prospective)

Problems with observation

Easy to misinterpret observational data

Regression to the mean (patients with extreme values tend to have less extreme values later, even without treatment)

Hawthorne effect (individuals modify their behavior in response to awareness of being observed)

Sampling

Data collection

Major challenges of observational studies

Selection bias: when sample population isn’t representative of target population

Confounders: must differ between the comparison group and predict the outcome of interest

• an unaccounted variable that ties exposure and outcome

How to reduce confounding in observational studies?

Design phase

• matching of controls on specific characteristics, restrict inclusion to certain categories of a confounder

Analytical phase

• Statistical adjustments: regression, propensity score, instrumental variable

• Sensitivity analysis of subgroups

Regression statistical adjustment

Estimates the association of each independent variable with the dependent variable after adjusting for the effects of other variables

Propensity score

A score that is the conditional probability of exposure to an intervention given a set of observed variables that may influence the likelihood of exposure

Estimate how likely its due to either exposure or to other factor

Instrumental variable

Pseudo-randomization method, divides the sample according to levels of a covariate that is associated with the exposure but not the outcome

Subgroup by variable linked to exposure but not outcome

What are residual confounders?

Unmeasured confounders

“unknown unknowns”

Factors we’re not aware of and can’t predict

most significant source of uncertainty and threat to validity of study

What is an interventional study?

Researcher usually determines/assigns who is exposed 

always prospective

What are the two main groups of interventional studies?

Controlled clinical trials: individuals are assigned to a group

Community (field) trials: entire groups (school, clinic, etc) are assigned to different interventions “cluster randomization)

Interventional studies (trials) can either be testing an intervention as … or ….

Preventative

Treatment

Non-randomized controlled trial overview

Doesn’t require a control group

Vulnerable to selection bias

Poor generalizability

Validity is usually LOW

Randomized trials: distribution

All factors (unknowns) that may impact outcome are assumed to be equally distributed among study groups

What is the best research design to establish causality?

Randomized controlled trials

What is the null hypothesis in RCT (superiority design)?

Treatment A and Treatment B are the SAME

Problem with simple randomization and solution

Can result in substantial imbalace

Solution: use blocking and/or stratified randomization

Stratification overview

Often used if there are planned subgroup analysis

Common ones: age, gender, race, country, presence/absence of a specific comorbidity

What describes the master plan for the research?

Study Protocol

What is the SOP

Standard Operating Procedures

Outlines how the study is executed/operationalized at the participant level (recruitment, retention, how measurements are taken, etc.)

What is key to internal validity in RCTs?

Standardization

Variability overview

High variability across study sites in the procedures to carry out the study make it more difficult to discern treatment differences.

If each site does it a different way, its harder to tell

What do the SOPs help reduce?

Measurement variability

How to reduce biologic variability?

Having adequate sample size and having repeated measures

Hard vs surrogate endpoints

Hard Endpoints: Overall survival, CV death, MI, stroke (life or death)

Surrogate: Change in BP, cholesterole, tumor size, HbA1c

What if the components of the composite are not all equally important?

Use a Win-Ratio analysis

What is a Win-Ratio Analysis?

statistical method used to evaluate treatment effects

Compares outcomes between treatment and control by forming pairs of patients (matched or random) and evaluating who “wins” based on predefined clinical hierarchy of outcomes

(its sorta like a war game)

Pros and Cons of Win-Ratio Analysis

Pros: Prioritizes clinical importance, avoids dilution (prevents less important outcomes overshawdowing critical ones like death)

Cons: Complex (requires careful planing), Limited adoption (not widely used outside of CV and perioperative trials)

What are secondary endpoints?

An endpoint that doesn’t constitute the basis of trial design

What is multiplicity?

Conducting multiple statistical analyses beyond the primary analysis

What can happen without adjustment for multiplicity testing?

Risk of false-positive is greatly increased

Why do traditional clinical trials often fail to translate into benefits for patients?

Design issues: surrogate, composite, and subjective outcomes (can exaggerate estimates)

Data issues: missing data

Selectively reported outcomes: Publication and reporting bias

Inappropriately interpreted outcomes

What are the main features of a pragmatic trial

Input from health system stakeholders

Intervention in routine clinical workflow

Data from the EHR

Diverse study population

Comparisons are real work (usually not placebo)

Outcomes important to decision makers

Explanatory vs Pragmatic Trial

Explanatory: Balance extraneous effects to study the “true” effects of the treatment, look for biologically meaningful results, highly selected patients with maximal probability to reveal a treatment effect. Ideal conditions

Pragmatic: extraneous effects are included, looking for results that are meaningful for decision making in routine clinical practice. Normal clinical conditions

Pragmatic Clinical trial vs RCT

Pragmatic: To improve practice and inform clinical/policy decisions. Meant for improving everyday clinical practice/decision making

RCT: To determine cause and effect under Ideal conditions

T/F No clinical trial is completely explanatory or pragmatic

True, they all fall somewhere inbetween

Internal vs external validity

high Internal = a well conducted study

high External = applicable to my patient, useful in practice

Noninferiority, what happens if CI crosses Delta?

Fails to reject the null

Noninferiority, what if study fails to reject the null?

could be that novel treatment is truly not non-inferior

The outcome rate in the SOC group was likely smaller than what authors assumed it would be

Which study uses a one-sided test? How about two-sided test?

One-sided: Noninferiority

Two-sided: superiority

Superiority vs noninferiority trials

Superiority

• Goal = different outcomes

• Null = 0 (no difference)

  • Reject null when CI does not cross 1 (OR/HR) or 0 (ARR)

Noninferiority

• Goal = not too much different

• Null => Delta (much worse)

  • Reject null when CI does not cross delta

What is a meta-analysis?

Systematic approach taking and analyzing data from multiple studies into a single effect estimate to answer a research question

An observational study of the evidence

Potential benefits of meta-analysis?

Confirm previous findings or clarify disparate results

Overall (pooled) effect estimate is statistically signficant but each individual (underpowered) study aren’t on there own

Could potentially reduce bias (but assuming only a minority of individual studies are biased and the majority aren’t)

Potential dangers of meta-analysis?

Could exacerbate bias (if majority of studies are biased)

Could provide false confidence

Could result in a Type 1 (or Type II) error

What is heterogenecity?

Level of differences between studies

The lower the heterogenicity, the more similar the studies are (homogenous)

What are the types of heterogenicity?

Clinical: population, intervention, comparator, outcomes

Methodological: Study design, procedure/protocol, quality, risk of bias (internal validity)

Statistical: Treatment effects of individual studies