HS

lecture 13

Decision Making for eHealth (COMP8160) - Week 15 (Part 2)

Importance of Prior Probability

  • Prior probability (prevalence) of an event in the population is crucial for diagnostic decision-making

  • Symptoms and published research results can help place patients into appropriate clinical subgroups where prevalence of specific diseases is known

The Three-Stage Diagnostic Process

Stage 1: Initial Judgment

  • Making an assessment about whether a patient likely has a disease

  • Establishes the prior probability (also called pretest probability)

  • Based on:

    • Epidemiological data

    • Clinical experience

    • Decision support systems

  • This is the P(A) term in Bayes' formula

Stage 2: Information Gathering

  • Involves collecting additional data through diagnostic tests

  • Primary goal: reducing uncertainty

  • Key factors in test selection:

    • Reliability of the test (sensitivity and specificity)

    • This stage presents significant opportunities for computational methods and eHealth applications

Stage 3: Probability Update

  • Updates the initial probability estimate based on test results

  • Calculates the posterior probability using Bayes' theorem

  • Ideally produces high posterior probability to increase confidence in diagnosis

  • This is the P(A|B) term in Bayes' formula

Measuring Test Performance

Confusion Matrix

  • Framework for evaluating diagnostic test performance:

    • True Positive (TP): Test correctly identifies presence of condition

    • True Negative (TN): Test correctly identifies absence of condition

    • False Positive (FP) (Type I error): Test incorrectly indicates presence of condition

    • False Negative (FN) (Type II error): Test incorrectly indicates absence of condition

    • Sensitivity = TP ÷ (TP + FN) = True Positives ÷ Total Condition Positive

    • Specificity = TN ÷ (TN + FP) = True Negatives ÷ Total Condition Negative

Bayes' Formula in Diagnostic Terms

  • Probability of disease given positive test:

    • P(A|B) = [P(B|A) × P(A)] ÷ P(B)

    • = [P(B|A) × P(A)] ÷ [P(B|A) × P(A) + P(B|A^c) × P(A^c)]

  • In clinical terms:

    • P(disease present | positive test) = [sensitivity × P(A)] ÷ [sensitivity × P(A) + (1 - specificity) × P(A^c)]

Practical Example: Heart Disease Testing

Case Study

  • Prior probability of heart disease in men with typical symptoms and family history: 0.95 (95%)

  • Exercise stress test characteristics:

    • Sensitivity: 0.65 (65%)

    • Specificity: 0.80 (80%)

Calculation

  • P(A|B) = [0.65 × 0.95] ÷ [0.65 × 0.95 + (1 - 0.80) × 0.05]

  • = 0.6175 ÷ (0.6175 + 0.01)

  • = 0.6175 ÷ 0.6275

  • = 0.98 (98%)

Impact of Improved Specificity

  • If specificity improved to 95%:

    • P(A|B) = [0.65 × 0.95] ÷ [0.65 × 0.95 + (1 - 0.95) × 0.05]

    • = 0.6175 ÷ (0.6175 + 0.0025)

    • = 0.6175 ÷ 0.62

    • = 0.996 (99.6%)

  • Higher specificity results in greater certainty with positive test results

Important Considerations in Diagnostic Testing

Limitations of Testing with Low Prior Probability

  • If prior probability is very low, a positive test result may only raise posterior probability to intermediate range

  • In these cases, positive test results alone may not provide sufficient confidence for diagnosis

Visualizing the Diagnostic Process

  • Positive test results generally increase probability of disease

  • Different tests have varying impacts on uncertainty reduction

  • Sequential testing can further refine probability estimates

Challenges and Limitations

  • Unreliable Prior Probability: If the initial estimate is inaccurate, Bayes' theorem will provide little value

  • Conditional Independence: When applying Bayes' theorem sequentially with multiple tests, tests must be conditionally independent

  • Violation of conditional independence leads to inaccurate posterior probabilities