2. Knowledge vs Expertise: the internet and the democratization of information

Multiple Sclerosis

• A neuro-degenerative, autoimmune disease

• Deterioration of myelin, the insulating material around nerves

• Symptoms include extreme fatigue, lack of coordination, weakness, tingling, impaired sensation, vision problems, bladder problems, cognitive impairment and mood changes

• Cause is unknown

• Canada has highest rates of MS in the world

Paolo Zamboni, MD

• Italian vascular surgeon

• 2009 “Chronic cerebrospinal venous insufficiency in patients with MS” J. Neur., Neurosurg. & Psych. 80 (4). (CCSVI)

• Hypothesis: blockages in neck veins (CCSVI) inhibits drainage of blood from brain, leaving behind iron deposits that are then attacked by immune system

• Proposed treatment: ‘Liberation Therapy’: angioplasty to open up neck veins to allow better drainage of blood from brain

Results of Zamboni’s study

• Looked at 65 patients diagnosed with MS and 235 controls (people without MS) • Found a 100% correlation between MS and CCSVI

Scientific Response

• An interesting –if somewhat incredible—hypothesis published in a lower quality journal

• However, little evidence that the mechanism proposed would explain MS

• Would require further study to be taken seriously

Liberation Therapy results

Zamboni published study on liberation therapy’s effectiveness as treatment for patients with relapsing-remitting MS (unblinded, no control)

• Some patients who underwent the operation reported and exhibited signs of improvement

• Media reports “Cause and Cure of MS Found”! (This was not Zamboni’s claim)

• Created demand among Canadian patients for treatment and to be covered by heath care system

• Neurologists and family doctors generally skeptical; Health Canada refused to cover the procedure: unproven, experimental, safety unclear

• Some MS patients paid out of pocket to travel to private clinics in other countries willing to do the operation

Reaction from MS Experts

• Is CCSVI the cause of MS?

• “X is the cause of Y” means:

• X is both necessary and sufficient for Y

• If you have X you have Y (X is a sufficient condition for Y)

• If you don’t have X you don’t have Y (X is a necessary condition for Y)

• But correlation is not causation: Does CCSVI cause MS or does MS cause CCSVI?

• Subsequent studies failed to replicate Zamboni’s results (far lower correlations found). CCSVI not the cause of MS.

On-line testimonials

• Anecdotal evidence: personal stories detached from proper scientific studies (i.e. randomized, controlled, doubleblind designed to limit biases, placebo effect, avoiding small sample size and nonrepresentative samples)

The impact of the internet and social media

• Patients who have the surgery and show signs of improvement post their success stories on-line and these are shared by others

• First cell statistics (anecdotal evidence)

• What do the other three cells look like?

• No one knew because these stories weren’t being shared on-line: people either embarrassed at having spent $$ for nothing or were positively discouraged (e.g. criticized on Facebook pages for being negative and unsupportive)

Placebo effect?

• People who expect to be improved by a pill often feel better (even if the pill contains no medicine at all, only sugar). Also works with sham operations.

• True vs false placebo effect. (Measurable physiological improvement vs merely putting a positive spin on how one feels)

• Expected remission stage? Post hoc ergo propter hoc? Some MS patients could be expected to experience remission of symptoms even without the operation—that’s normal pattern for many

The post hoc ergo propter hoc fallacy:

‘After this, therefore because of this’

This is why scientists perform ‘blind’ studies – to control for the placebo or coincidental effects—participants don’t know whether they received the treatment or a placebo.

In a double-blinded study, the scientists performing the experiment don’t know which participants receive the treatment or placebo either (so they don’t accidentally tip the patient off and bias the study); but the scientists analysing the results do.

They can then see whether people in the treatment group saw greater improvement than those in the control group.

Coincidental effect?

• The angioplasty may make some people feel better without actually addressing the underlying cause of their MS symptoms • Compare: Replacing your air filter may make your car run slightly better—even though the real problem is old spark plugs or battery etc

Experts’ concerns:

1) Safety of procedure (for MS) unproven. 3 deaths associated with the liberation procedure

2) No one will volunteer for a proper clinical study (randomized blinded controlled) if they might not get the ‘real’ procedure

3) Funds will be diverted from studying more promising lines of research, e.g. stem cells

4) Sets precedent for other patient groups that by making enough noise they too can control the direction of medical research—point the flashlight in a particular direction (more ‘democratic’ perhaps but better results?)

Results of subsequent studies

• 2013 Lancet study: CCSVI just as common in control group as those with MS

• 2% with MS had CCSVI; 3% in healthy control.

• 2017 Canadian study: Liberation therapy provided no significant improvement over a placebo (sham-operation) in a double-blind study—neither patients nor physicians evaluating the results knew which patients received liberation therapy as opposed to having a shunt inserted in neck with no balloon angioplasty.

• Zamboni himself publishes similar results after conducting randomized controlled study in Journal of the American Medical Association (JAMA) 201

Reactions of (some) Liberation Therapy advocates:

• Neurologists trying to protect their turf from vascular surgeons—motivated by $

• Neurologists in business with big pharma (pushing pills)—motivated by

• Conspiracy thinking: all the doctors and scientists criticizing Liberation Therapy are part of a massive cover-up of the truth; they want to keep patients sick for their own benefit ()

• Assumption of conspiracy theory thinking: there are no reasonable grounds for disagreeing with me—anyone who does not think as I do must be biased or part of the conspiracy. This is an unfalsifiable belief (i.e. immune to all and any evidence).

• There are legitimate reasons to criticize the pharmaceutical industry. However…

quote from author of Bad Pharma and Bad Science

Problems with pharm don’t mean that quackery works. problems in aircraft design do not mean that magic carpets can actually fly

Effects of the internet

• The ‘democratization of information’

• Scientific studies no longer the special privilege of scientific elites

• Ordinary citizens can inform themselves about science and new breakthroughs—”do their own research”

• But are they capable of understanding what they read or of actually doing proper unbiased research

I did1 my own2 research3

1 watched 2 someone elses 3 shitty youtube video

Populism

these smug pilots have lost touch with regular passengers like us. Who thinks I should fly the plane?

Is having information the same as having expertise?

• What is the difference?

• Expertise requires: • Understanding of how the information (evidence, hypotheses, trials) is created • Understanding a study’s limitations and the merits of alternative studies • Not just ‘cherry-picking’ studies that confirm your opinion

• Science not just as product but as process— how to actually do science (knowing how, not only knowing that)

• Science as a ‘way of knowing’ aims at objective knowledge, not just confirming one’s beliefs and values (Mertonian values)

• ‘Google University’ doesn’t teach that