Logistic Regression

When is logistic regression used?

When the response variable is categorical, usually binary (Yes/No, Success/Failure, 0/1)

Why not use linear regression for a binary response?

Linear regression can predict values below 0 or above 1 and does not model probabilities appropriately.

What does logistic regression predict?

Probability of success (or being in a particular category)

What is probability?

The chance an even occurs between 0 and 1

If probability = 0.80, what does that mean?

80% chance of success.

If probability = 0.10, what does that mean?

10% chance of success.

What are odds?

Successes relative to failures.

Formula for odds?

Odds = Probability / (1 − Probability)

Probability = 0.80. What are the odds?

0.80 / 0.20 = 4

Probability = 0.25. What are the odds?

0.25 / 0.75 = 1/3 ≈ 0.33

What do odds = 4 mean?

Success is 4 times as likely as failure.

What are log-odds?

The logarithm-odds

Why does logistic regression use log odds?

Because log-odds can take any value from negative infinity to positive infinity, making modeling easier

Do we usually interpret coefficients directly as probabilities

No

What does a positive coefficient mean?

As X increases, probability of success increases

What does a negative coefficient mean?

As X increases, probability of success decreases

What does a coefficient of zero mean?

No relationship between X and the response

What does a coefficent test in logistic regression examine?

Whether a predictor is useful.

Null hypothesis?

β₁ = 0

Alternative hypothesis?

β₁ ≠ 0

---

If p < 0.05?

Reject H₀. Predictor is useful.

If p > 0.05?

Fail to reject H₀.

Predictor coefficient = 0.8. What direction is the relationship?

Positive.

Predictor coefficient = -1.2. What direction is the relationship?

Negative.

Significant positive coefficient means?

Increasing X increases the probability of success.

Significant negative coefficient means?

Increasing X decreases the probability of success.

When is multinomial logistic regression used?

When the response has more than two categories.

What is an example of multinomial Logistic Regression categories?

Freshman / Sophomore / Junior / Senior

What is a baseline category?

The reference category to which other categories are compared.

How are coefficients interpreted?

Relative to the baseline category.