Probability, Bayes' Rule, and the Foundations of Hypothesis Testing

Probabilistic Concepts and Long-Run Frequency

• Probability is defined as the idea of long-run frequency. For example, if a coin is tossed 100 times, one would expect to get heads roughly 50% of the time.

• In health contexts, probability is used to determine risks, such as how many people would get food poisoning if 100 were exposed to contaminated food.

• Probability values are strictly bounded between $0$ (the event never happens) and $1$ (the event always happens).

• While probabilities are often expressed as percentages for ease of communication, calculations must always utilize proportions (e.g., $0.01$ rather than $1\%$) to ensure mathematical accuracy.

Conditional Probability and Interpretive Risks

• Conditional probability involves limiting the denominator to a specific subset of a population. It is expressed using a "pipe" symbol, which means "given."

• A common point of confusion in conditional probability is the inversion of conditions. Two quantities that sound similar can be vastly different:

  • $P(\text{Being a man} | \text{Driving a Porsche})$: Historically high, as most Porsche owners in specific clubs were male.

  • $P(\text{Driving a Porsche} | \text{Being a man})$: Very low, as very few individual men own Porsches.

• This principle is critical in cardiovascular disease (CVD) risk assessment:

  • The probability of being high risk, given that you have CVD ($P(\text{High Risk} | \text{CVD})$), is high.

  • The probability of having CVD, given that you are high risk ($P(\text{CVD} | \text{High Risk})$), is relatively low.

The OJ Simpson Trial: A Case Study in Conditional Probability

• In 1994, Nicole Brown was murdered, and her husband, OJ Simpson, was the prime suspect. Simpson had a history of abusing his wife.

• The defense lawyer argued that only $0.1\%$ of men who physically abuse their wives eventually murder them, implying the history of abuse was irrelevant.

• The statistical error lay in the condition. The relevant probability for the court was not based on all abusive relationships, but specifically on cases where the wife was already murdered.

• Comparative Probabilities:

  • $P(\text{Murder} | \text{Abuse}) \approx 1 \text{ in } 1000$.

  • $P(\text{Guilt} | \text{Abuse and Spouse Murdered}) \approx 50\%$.

• Following proper probabilistic logic, the realistic probability of guilt in such a specific scenario is estimated to be near $80\%$.

Fundamental Rules of Probability

• Multiplication Rule: Used for independent events. For example, the probability of having two boys in a row is calculated as $0.5 \times 0.5 = 0.25$.

• Additive Rule: Used to find the chance of either of two events occurring. To avoid double-counting the overlap (joint probability), the formula is used:

  • $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$.

  • Example: The chance of at least one child being a boy in a two-child family is $0.5 + 0.5 - (0.5 \times 0.5) = 0.75$.

• Airport Screening Example: If one diagnostic test has a $70\%$ sensitivity ($0.7$ probability of positive test given disease), doing two tests increases the detection rate to $91\%$:

  • $0.7 + 0.7 - (0.7 \times 0.7) = 0.91$.

• Joint Probability: Calculated by multiplying the conditional probability of A given B by the risk of B:

  • $P(A \cap B) = P(A | B) \times P(B)$.

Diagnostic Testing: Accuracy and Predictive Value

• There are four primary measures used to evaluate diagnostic tests:

  1. Sensitivity: $P(\text{Test Positive} | \text{Disease})$. This measures the accuracy of the test in identifying those who are ill.

  2. Specificity: $P(\text{Test Negative} | \text{No Disease})$. This measures accuracy in identifying those who are healthy.

  3. Positive Predictive Value (PPV): $P(\text{Disease} | \text{Test Positive})$. This is the probability the doctor is most interested in.

  4. Negative Predictive Value (NPV): $P(\text{No Disease} | \text{Test Negative})$.

• The Paradox of Diagnosis: Sensitivity and specificity are inherent to the test itself (often provided by manufacturers). However, PPV depends heavily on the prior probability (prevalence) of the disease in the population.

• If a disease is extremely rare, even a positive result from a highly sensitive test may still result in a low PPV.

Bayes' Rule and The Likelihood Ratio

• Reverend Thomas Bayes developed the formula to invert probabilities, allowing clinicians to go from sensitivity/specificity to PPV.

• Likelihood Ratio (LR): This is the ratio of true positives to false positives.

  • $\text{LR} = \frac{\text{Sensitivity}}{1 - \text{Specificity}}$.

  • If a test has an LR of $3$, a person who tests positive is three times more likely to be a true positive than a false positive.

• Simplified Bayesian Rule of Thumb:

  • $\text{PPV} \approx \frac{\text{Prevalence} \times \text{Sensitivity}}{1 - \text{Specificity}}$.

• Fagan's Nomogram: A visual tool representing how prior probability is converted to posterior (post-test) probability using the Likelihood Ratio (a straight line connecting the two).

The Scabies Case Study: Sensitivity vs. Specificity

• In public health, a choice must be made between sensitivity and specificity.

• For scabies, specific signs (like seeing mites or burrows) are rare. A dermatologist focused only on specific signs will miss many cases (low sensitivity).

• A sensitive approach treats anyone with symptoms. While this includes some false positives, the treatment is harmless and one-off, preventing further transmission by those who might otherwise have been missed.

• The clinical aphorism "A dermatologist is not a dermatologist until they have missed scabies" implies that clinical screening for scabies is often not sensitive; many visits may be required before a definitive diagnosis is made.

Frequentist Statistics and Hypothesis Testing

• Frequentist statistics is built on the concept of repeating an experiment over and over again (long-run frequency), even if the study is only conducted once.

• Historical Origins: Modern statistics emerged in the 1930s from agriculture. R.A. Fisher demonstrated that factors like rainfall had a much larger impact on crop yields than different fertilizers, identifying massive confounding.

The World's First Hypothesis Test: The Lady Tasting Tea

• Muriel Bristol claimed she could distinguish whether milk or tea was added to the cup first. R.A. Fisher designed an experiment to test this claim.

• Experimental Design: 8 cups were provided (4 milk first, 4 tea first). There were $70$ possible combinations of choices.

• The Null Hypothesis ($H_0$): The lady has no ability to tell the difference and is merely guessing.

• Distribution of Probabilities (under $H_0$):

  • Chance of getting $0$ out of $4$ correct: $1 / 70$.

  • Chance of getting $2$ out of $4$ correct (most likely): Higher probability.

  • Chance of getting $4$ out of $4$ correct: $1 / 70$ (approximately $1.4\%$).

• Because getting all 4 correct was so unlikely to happen by luck ($p < 0.05$), Fisher rejected the null hypothesis and concluded she could truly tell the difference.

P-Values and Significance

• Definition: A p-value is the probability of the observed results, or more extreme results, occurring if the null hypothesis is true.

• It is a conditional probability: $P(\text{Data} | H_0)$, not the other way around.

• Convention: If $p < 0.05$ ($1$ in $20$), the result is deemed "statistically significant," and the null hypothesis is rejected in favor of an association between exposure and disease.

• Reporting: Computer software often rounds p-values to $0.000$. However, a p-value can never be zero; it should be reported as $p < 0.001$.

Confidence Intervals

• Definition: A range of values compatible with the true population value.

• Frequentist Interpretation: If the study was repeated 100 times and an interval was calculated each time, $95$ of those $100$ intervals ($19$ out of $20$) would contain the true value.

• Alternative Interpretation: The 95% Confidence Interval provides a range of values that are statistically compatible with the real-world effect.

Measures of Association: Risk Ratios

• Risk Ratio (RR): Compares the risk of an outcome in an exposed group to the risk in an unexposed group.

• Smoking and Lung Cancer Example:

  • If the risk of lung cancer is 5 times higher in smokers than non-smokers, the $RR = 5$.

• Electric Toothbrush and Caries Example:

  • Calculated $RR = 0.23$. This indicates a $77\%$ reduction in caries ($1 - 0.23 = 0.77$).

  • If the confidence interval includes the value of $1$ (no effect), the result is not statistically significant.

The Chi-Square Test (\chi^2)

• Used for testing associations between two categorical variables.

• Expected Value: The value expected in a table cell if there was no association between exposure and outcome. Formula:

  • $\text{Expected Value} = \frac{\text{Row Total} \times \text{Column Total}}{\text{Grand Total}}$.

• The Chi-Square statistic is the sum of the squared differences between observed ($O$) and expected ($E$) values:

  • $\sum \frac{(O-E)^2}{E}$ .

• If the difference is large enough (exceeding a threshold like $3.84$ for a 2x2 table), it results in a low p-value, indicating the observed results are unlikely by chance.