Module Recap and Exam Preparation Notes

Recap of Module Topics

• Today's lecture will include a summary of all topics covered in the module.
• Encourage students to complete homework before Thursday's discussion.
• Students will be shown a past exam paper and will work on two specific questions until Thursday.

Session Structure

• The upcoming session on Thursday will be student-led to ensure engagement and understanding.
• Emphasizes the importance of student effort in preparation.

Overview of Medical Data

• Medical data is crucial in healthcare delivery, including in primary care, hospitals, and referrals.
• Functions of medical data:
• Categorizing problems.
• Understanding disease development and spread.
• Supporting decisions on treatments.
• Types of medical data:
• Narrative text data (historically dominant).
• Numerical measurements (e.g., blood pressure, glucose levels).
• Signal recordings (e.g., EEG).
• Images (e.g., MRI scans).

Difference Between Data, Information, and Knowledge

• Data: Raw entities or values (e.g., temperature, medical history).
• Information: Interpretation of data (e.g., determining if a temperature reading is high).
• Knowledge: Insights gained through reasoning, studies, and comparisons of data (e.g., link between high sugar levels and diabetes risk).

Data Mining Process

• Definition: Extracting meaningful information from data.
• Knowledge Discovery in Databases (KDD) process:
• Raw data collection, pre-processing, data mining (clustering/classification), and evaluation.
• Emphasis on structured data: rows (records) and columns (attributes).

Data Mining Tasks

• Differentiation between descriptive and predictive data mining tasks.
• Classification: Learning patterns to predict outcomes.
• Training set: Known outcomes used to build the model.
• Test set: Unseen data where outcomes are predicted based on the model.
• Evaluation metrics: confusion matrix (true positive, false positive, true negative, false negative) for model accuracy.

Model Evaluation

• Accuracy calculation:
  Accuracy = \frac{TP + TN}{TP + TN + FP + FN}
• Sensitivity (true positive rate) and specificity (true negative rate) also calculated to assess model performance, especially in imbalanced classes.
• Trade-offs: Adjusting thresholds can impact sensitivity and specificity.

Importance of Interpretability in Classification Algorithms

• White box models (e.g., decision trees) vs. black box models (e.g., neural networks).
• Preference for white box models in healthcare for clinician understanding.

Clustering Techniques

• Definition: Grouping similar data points based on characteristics, no target variable ('unsupervised learning').
• Examples:
• Hierarchical clustering (building dendrograms).
• K-means and PAM (Partitioning Around Medoids) clustering (distance-based clustering).
• Fuzzy C-means (allowing overlap between clusters).

Decision Making Under Uncertainty

• Introduction to Bayes' Theorem for updating probabilities based on evidence.
• Relation to prior probabilities, sensitivity, and specificity.
• Application in a multi-test scenario to improve diagnostic accuracy.

Key Takeaways and Homework

• Students encouraged to review materials, especially case studies or examples covered in class.
• Homework:
• Download and review the 2021 past exam paper from Moodle.
• Attempt questions three and four for Thursday's session discussion.