Lec 6: Case study on the impact of hope and hype on descion making

True or false: some medical information is flawed?

true - too many reserach studies are poorly designed or executed - there is a flawed experimental design or how the reserachers conducted study (maybe because they deviated from the initial plan)

What common flaw occurs when research is pushed to finish too quickly?

Early studies often lack control groups or include too few participants, making their conclusions unreliable.

Molnupiravir

An oral antiviral drug developed at Emory University and licensed to Merck; designed to disrupt viral RNA replication.

• A treatment hailed as revolutionary before being fully validated — often generating public and policy hype.

• it forces whatever virus is in the body to form a bunch or mutations rendering their reproduction

What was molnupiravir (Lagevrio) approved for?

As an oral antiviral treatment for at-risk, non-hospitalized adults with mild to moderate COVID-19.

What did early clinical trials claim about molnupiravir’s effectiveness?

It reportedly reduced the risk of hospitalization or death by 50%.

• regulators claimed it was safe and effecrive and an importnat tool in fighting COVID-19

• urgent need and optimism can lower evidence standards, leading to potentially premature regulatory approval.

It reportedly reduced the risk of hospitalization or death by 50%.

Because approval was granted quickly, based on limited or early trial data, before full peer-reviewed evidence was available.

What type of studies supported molnupiravir’s approval at the time?

Mostly preprints — early study manuscripts that had not yet been peer-reviewed, meaning their conclusions could still change.

• its risky to rely on preprints because they have not been critically reviewed by experts, and errors or overstatements may go unchecked before influencing policy.

• also only  early-phase, small-scale trials with limited participant numbers, lacking long-term safety data had been released.

What were the primary outcomes measured in the early molnupiravir trials?

The studies primarily assessed reduction of viral RNA and infectious virus in the nasopharyngeal area (between the nose and lungs).

• but they didnt evaluate clinical outcomes like symptom severity and/or hospitilization

What did the authors themselves acknowledge as a limitation?

That their study focused only on antiviral activity (physiological effects- like viral load), not clinical efficacy or patient-centered outcomes.

Why is this distinction important for policy and approval decisions?

Because reducing viral load doesn’t always mean the drug improves patient outcomes — policies based on such evidence may overstate real-world benefits.

What was still ongoing at the time of the initial report?

A Phase 2/3 study to evaluate molnupiravir’s effects on symptom duration, severity, emergency visits, and hospitalization.

Why was molnupiravir’s approval controversial?

Because the UK approved it while the key trial (MOVe-OUT) was still unfinished and unpublished — meaning the evidence wasn’t yet verified.

• MOVe-OUT was the name of the main clinical trial that tested molnupiravir (Lagevrio) in humans with COVID-19.

Criticism of the MOVe-OUT trial

• Selection bias in the study population (how patients were chosen).

• Incomplete or unclear patient data and follow-up.

• Weak or flawed statistical analysis that might have exaggerated the drug’s benefit.

group from mcmaster pointed these out

What happened in the second subgroup of trial partciapnts?

The apparent benefit disappeared — the second half of participants showed a 35% increase (not decrease) in hospitalization or death risk, meaning no significant benefit overall. it hsowed that placebo patients acctually fared slightly better. 

• The first half of the trial made it seem as though the drug reduced the risk of hsopitilization or death by 50%.

• the problem was that the claim of efficacy was based on an analysis of only half the planned participants - therefore half a dataset. 

Lesson of molnupirivar trial 

never change policies based on a subset of halfset of data - could be smaller dataset that doesnt accurately represent populations

• It shows how early, incomplete data can exaggerate a drug’s effectiveness, especially under pressure for quick results.

What did the final published MOVe-OUT report show about molnupiravir’s effectiveness?

The combined results showed only a 3% reduction in hospitalization or death due to COVID-19 — much smaller than the early claim of 50%.- the revised claim: “potentially meaningful for patients, health systems, and public health”

What did Merck do during the MOVe-OUT trial that caused concern?

They stopped enrolling new participants early after positive interim results, leading to a smaller sample size than originally planned.

• It increases the chance of a false positive — the drug may appear effective just by chance, not because it truly works.

What did Merck originally state in their trial protocol?

That there were no plans to discontinue the study early, even if early data appeared positive.

• they changed their protocol which is considered methodologically improper and unprecedented in public record.

What is an interim analysis?

It’s when researchers examine trial data before the study is finished to decide whether to stop, modify, or continue the trial.

• drawback is that there is more of a false positive risk 

Why do companies perform interim analyses?

Often for financial reasons — to save money if early results look good (or bad), or to push for faster approval of a promising drug.

How can interim analyses increase the risk of false positives?

Each time data are checked early, it raises the chance that random variation will look like a real effect — meaning “lucky flukes” can be mistaken for true results.

P-value Hacking / Fishing

the process of repeated checking data or testing different endpoints until a statistically significant result is found, inflating the risk of reporting finds that are not reproducible. 

What major issue did critics identify in the MOVe-OUT trial data?

There was an unexplained shift — the early results showed a 50% reduction in hospitalization or death, but the later results showed no benefit and even a trend favoring the placebo group

• why 50% to 3%

• They did not provide any analysis or rationale for how or why the treatment effect flipped so dramatically between the first and second halves of participants.

Why is this reversal suspicious?

Because such a strong early/late split is very unlikely to occur by chance, suggesting possible selection bias or changes in how participants were enrolled.

What is “selection bias”?

When differences in how participants are chosen or grouped affect results — for example, one group may end up being healthier or sicker on average.

• It’s a type of bias that occurs when the participants included in a study are not representative of the overall population being studied.

What are some common types of selection bias?

• Sampling bias: Certain groups in the population are less likely to be included than others.

• Attrition bias: Participants who drop out of the study differ from those who stay in.

• Volunteer bias: People with specific characteristics (like higher health awareness or income) are more likely to participate.

• Non-response bias: People who don’t respond or refuse to participate differ systematically from responders.

What does “lost to follow-up” mean in a clinical trial?

It refers to participants who stop responding or can’t be contacted during a study, so their outcomes are unknown.

How did MOVe-OUT researchers handle “lost-to-follow-up” patients?

They assumed those participants had died and counted them as primary events (deaths) in the analysis.

• It’s methodologically inappropriate because missing data doesn’t mean a participant experienced the worst outcome — they may have simply moved or stopped answering calls

What effect did this assumption have on the trial results?

It inflated the apparent risk reduction, making the drug seem more effective than it truly was.

What did Thorlund et al. show would happen if this bias was corrected?

The relative risk (RR = 0.70) would become non-significant, meaning the drug’s benefit might disappear statistically.

What eventually happened to molnupiravir’s approval in some regions?

It was reconsidered or withdrawn after further large clinical trials (such as the PANORAMIC trial) found minimal benefit.

• it was completely disproven 

What is the main lesson from this policy reversal?

A large-scale follow-up study in the UK that re-evaluated molnupiravir’s real-world effectiveness, showing limited benefit for most patients.

Who conducted the PANORAMIC trial and how many participants were there?

The trial was conducted at Oxford University with 25,783 participants, making it more than 20× larger than the MOVe-OUT trial.

• main goal to determine whether molnupiravir, when added to usual care, reduced hospitalizations or deaths among high-risk adults with COVID-19.

• It found that molnupiravir did not reduce the frequency of hospitalisation or death among high-risk vaccinated adults in the community.

• It had a much larger sample size, real-world participants (many vaccinated), and was independently run, providing more reliable and generalizable data.

Glybera (alipogene tiparvovec)

The first approved gene therapy (Europe, 2012) for a rare fat metabolism disorder; withdrawn in 2017 after treating only one patient commercially due to ineffectiveness and high cost (~$1 million per dose).

• It shows how premature optimism and hype around “revolutionary cures” — especially for rare diseases — can lead to high costs, limited use, and eventual withdrawal when real-world results don’t meet expectations.