diagnostic statistics

What is the purpose of diagnostic statistics?

Help us in situations where we are predicting a categorical outcome.

What is a confusion matrix?

A table that cross-tabulates predicted vs. actual categories to evaluate prediction accuracy; usually visualized as a 2x2 matrix for binary outcomes.

What are the four main components of a 2x2 confusion matrix?

True Positives (TP), False Positives (FP), True Negatives (TN), False Negatives (FN).

What is sensitivity?

Also known as true positive rate or recall (tells us what percent of cases we correctly identified):
Formula: TP / (TP + FN)

What is specificity?

Also known as true negative rate (tells us what percent of non-cases we correctly identified):
Formula: TN / (TN + FP)

What is precision (positive predictive value)?

The proportion of predicted positives that are correct (tells us what percent of predicted cases ended up being correct):
Formula: TP / (TP + FP)

What is negative predictive value?

The proportion of predicted negatives that are correct (the percent of predicted non-cases that ended up being correct):
Formula: TN / (TN + FN)

What is percent accuracy?

Overall correct predictions:
Formula: (TP + TN) / (TP + TN + FP + FN)

What does the kappa statistic measure?

Penalizes our percentage to account for how much we would have expected to get right just by chance (i.e., if we were totally guessing).

What is balanced accuracy?

The arithmetic mean of sensitivity and specificity.

What is the F1 score?

The harmonic mean of sensitivity and positive predictive value

What is class imbalance?

When one category has significantly more cases than the other, making accuracy less meaningful.