Lecture 29/30 - Critical Appraisal of a Non-Inferiority Trial

what are additional questions to ask when evaluating a non-inferiority trial

did the investigators guard against unwarranted conclusion of non-inferiority?

was the effect of the standard treatment preserved?

did the investigators analyze patients according to the treatment received and to the groups to which they were assigned?

what is a superiority trial

a study to prove one intervention is better than the other

often superiority vs placebo

what is the null hypothesis for a superiority trial

start by assuming there is no difference between groups

what is the alternative hypothesis for a superiority trial

suggests that there is a difference between groups

what is a type 1 error

probability of concluding that there was a difference, when there was no difference

i.e. false positive

what is the probability of making a type 1 error

alpha

arbitrarily set at 0.05 → p < 0.05

what is a type 2 error

probability of concluding there was no difference when there was a difference

i.e. false negative

what is the probability of making a type 2 error

beta

what is the main determinant of making a type 2 error

sample size

what are the 3 possible outcomes of a superiority trial

what are problems with superiority RCTs

unethical to do a placebo-controlled trial

regulatory agencies want clinical endpoint trials, not surrogate endpoint trials

incremental benefits of therapies are getting smaller

may want to prove that a new drug is efficacious and: safer, costs less, more convenient, improves quality of life

what is an example of a research question for a superiority trial

with a 95% certainty can I say that the mortality rate of the study drug is 30% lower than that of the control?

what is an example of a research question for an equivalence trial

with a 90% certainty can I say that the AUC for a generic formulation of a drug lies within 80% to 125% of the AUC for the brand name standard?

what is an example of a research question for a non-inferiority trial

with a 95% certainty, can I say that Drug A is no worse than 30% than Drug B?

what is the purpose of equivalence trial

aim to rule out differences between two treatments

this difference is outside a zone of scientific or clinical indifference

most commonly done in the context of establishing bioequivalence of generic medications

what is the goal of non-inferiority trials

designed to prove that one intervention is not worse than another treatment by some pre-specified amount

• amount of benefit you are willing to sacrifice to use the new drug instead of the old drug

where are they used: CVOT in diabetes, novel oral anticoagulants in atrial fibrillation, antibiotic therapy, new inhaled therapies for COPD

what is the null hypothesis of a non-inferiority trial

there is a difference between groups

what is the alternative hypothesis for a non-inferiority trial

the difference in effect between the new and old interventions is less than the pre-specified amount

what is a type 1 error of a non-inferiority trial

deciding a treatment is non-inferior when it is inferior

i.e. false negative (opposite of normal type 1 error)

what is a type 2 error of a non-inferiority trial

deciding a treatment is inferior when it is actually non-inferior

i.e. false positive (opposite of normal type 2 error)

what else can an non-inferiority trial show

can show superiority

must test for non-inferiority first, then superiority

what are the 3 interpretations if non-inferiority is shown

1. both drugs are equally effective (not worse than)

2. both drugs are equally ineffective

3. the trial was insufficiently designed to find a difference

how should a failed superiority trial be interpreted

negative superiority trials do not demonstrate that therapies are equivalent

lots of time the trials are not large enough to show equivalence

what is the risk of a poorly done non-inferiority trial

falsely suggest there is no difference between groups → false negative trial and drug put into practice

how can investigators guard against an unwarranted conclusion of non-inferiority

1. ensure the active control was previously proven to be effective

2. ensure study patients and outcomes are similar to those in the original trials establishing efficacy of the active control

3. ensure both regimens were applied in optimal fashion

4. ensure patients are analyzed according to the treatment they received and to the groups they were assigned

5. ensure the non-inferiority margin is pre-specified

6. ensure sample size is large enough

what is the issue of assay sensitivity seen in non-inferiority trials

in some disease settings even truly effective drugs do not always show a benefit in clinical trials

assay sensitivity = the ability to distinguish effective from ineffective therapies

what are some ways that investigators can produce apparent non-inferiority

enrolling patients less likely to be adherent

enrolling patients less likely to be responsive to standard treatment

enrolling a low-risk population

reducing treatment intensity

administering treatment by a suboptimal route

terminating follow up before treatment effects manifest

what is the effect of using intention to treat analysis in a superiority trial

underestimates treatment effect, favours finding no difference

what is the effect of using per protocol analysis in a superiority trial

overestimates treatment effect, favours finding a difference

what is the effect of using intention to treat analysis in a non-inferiority trial

underestimates treatment effect, favours finding non-inferiority

what is the effect of using per protocol analysis in a non-inferiority trial

overestimates treatment effect, favours finding against non-inferiority

how to ttell a study has a non-inferiority goal

look for the non-inferiority statement: this intervention is not that much worse than the control

how is a pre-specified margin set in a non-inferiority trial

main approaches: statistical method, clinical method

margin may be set using: absolute differences, relative differences

the margin should be based on sound statistical reasoning and clinical judgement

should be: defined a priori, justified clinically, must be smaller than the difference between gold standard and placebo in superiority trials

what is the smallest acceptable degree of clinical inferiority of the new treatment

at least 50% of the minimal treatment effect of the active control demonstrated in prior meta-analysis be preserved

OR or RR of 1.15 to 1.20 has been commonly used as a clinically acceptable margin

why is it important that the sample size is large enough in a study

failure to find a difference when one really exists (type 2 error) may be due to lack of power because of small sample size or low event rates

in non-inferiority trials, the smaller the margin on non-inferiority, the more study subjects and events needed to show non-inferiority

what is the sample size of a study dependent on

power (increased power = increased sample size)

type 1 error rate (alpha, usually 0.05 but smaller = greater sample size)

estimated difference you think you will detect (smaller difference = greater sample size)