Biostats - ROC curves and logistic regression

What does binary logistic regression predict?

The probability of a binary outcome (success=1 or failure=0).

How is the dependent variable different from linear regression?

It is binary, not continuous.

Examples of binary DV responses?

Yes/no, alive/dead, has/doesn't have, increased/decreased.

Why is logistic regression widely used?

Useful for research questions based on dichotomies (medicine, psych, sociology, etc.).

What does logistic regression try to predict mathematically?

p(success) = f(covariates).

What assumption must success/failure categories meet?

They must be mutually exclusive and collectively exhaustive.

How must success/failure be coded in logistic regression?

Failure=0, success=1.

Why can’t linear regression be used to predict probabilities?

Linear regression can produce predictions outside the 0–1 range.

What ensures logistic regression predictions stay between 0 and 1?

A link function.

What link function does binary logistic regression use?

The logit function.

What is a Generalized Linear Model (GLM)?

A regression model that uses a link function.

What does the logit function represent?

The natural log of the odds of success.

Instead of b-coefficients, what do we usually interpret in logistic regression?

Odds ratios (Exp(B)).

What does an odds ratio represent?

Change in odds of success for a 1-unit change in the predictor.

How do you compute an odds ratio from b?

Odds ratio = e^b.

What does H₀ say about the odds ratio?

H₀: Odds ratio = 1 (no effect).

If odds ratio > 1, what does it mean?

Positive relationship; higher covariate increases odds of success.

If odds ratio < 1, what does it mean?

Negative relationship; higher covariate decreases odds of success.

Example: If b = 2.69, what is the odds ratio?

14.73 (e^2.69).

Interpretation of odds ratio = 14.73?

The odds of success are 14.73 times higher for a 1-unit increase in predictor.

How do you compute odds ratio for a non-unit change?

Multiply b by the change size and exponentiate: e^(b * change).

If b = 2.69, what is OR for a 2-unit increase?

e^(5.38) = 217.02.

How do multiple predictors combine in logistic regression?

Their effects multiply on the odds.

How do effects combine in linear regression?

They add on the outcome.

Why is this difference important?

Logistic = multiplicative effects; Linear = additive effects.

What shape do predicted probability curves form?

An S-shaped logistic (sigmoid) curve.

Summary: What does logistic regression do?

Predicts probability of success/failure using a logit link and odds ratios.

What determines classification in logistic regression?

Predicted probability of success.

What is the default cut value for classifying success?

p(success) > .50.

What is a classification table?

A confusion matrix of predicted vs observed outcomes.

What does the classification table show?

Correct successes, correct failures, false positives, false negatives.

Examples of performance measures derived from it?

Sensitivity, specificity, PPV, NPV.

Why can’t you minimize all classification errors?

Reducing false positives usually increases false negatives (trade-off).

What is the Null model in logistic regression?

A model with no predictors.

What does the Null model predict for every case?

The most common outcome category.

Why does it always predict the largest category?

It minimizes overall classification error without predictors.

Why are rare events a problem in logistic regression?

The Null model predicts “does not occur” and is mostly correct, leaving little room for improvement.

How do you decide which covariates to include?

Check if success/fail groups differ on continuous variables (mean differences) or categorical variables (chi-square differences).

How do you expand a logistic model?

Add predictors and compare classification tables.

Why do we need ROC curves?

They show model performance for different cut values (not just .50).

What is an ROC curve?

A plot of sensitivity (TPR) vs 1–specificity (FPR) across cut values.

Where did ROC curves originate?

WWII radar detection.

What does sensitivity represent?

True positive rate.

What does specificity represent?

True negative rate.

What happens with a low cut value?

High sensitivity, more false positives.

What happens with a high cut value?

High specificity, more false negatives.

What is the purpose of comparing ROC curves?

To see which model achieves better sensitivity-specificity trade-offs.

What is the AUC (Area Under the Curve)?

An overall measure of diagnostic accuracy.

What does AUC close to 1 mean?

Model distinguishes successes from failures very well.

What does AUC = .50 mean?

Model is no better than chance.