logistic regression

when to use

• outcome (y) is a category, not a number

• Binary: 0 or 1, Yes/No, Buy/Not Buy.

• The model predicts probability of the event (y=1) occurring.

fitting

• fit <- glm(y ~ x1 + x2, data=df,

• family=binomial(link="logit"))

• summary(fit)

• Uses glm() not lm(). The family=binomial argument is what makes it logistic.

predictions

• predict.glm(fit, newdata=new_df, type="response") # probability (0-1)

• predict.glm(fit, newdata=new_df, type="link") # log-odds

using to classify

• probs <- predict.glm(fit, newdata=df, type="response")

• predicted <- ifelse(probs >= 0.5, "Yes", "No")

evaluating model

• library(caret)

• confusionMatrix(predicted_factor, actual_factor)

• Gives accuracy, sensitivity (true positive rate), specificity (true negative rate), precision. A model can have high accuracy but be bad at one class.

interpreting coeff.

• exp(coef(fit)) # convert log-odds to odds ratios

• Coefficients are in log-odds (hard to interpret directly). After exp(): a value of 1.5 means the odds of y=1 are 1.5× higher per unit increase in x. Sign tells direction: positive = increases probability, negative = decreases it.

multinomial

• library(nnet)

• fit <- multinom(y ~ x1 + x2, data=df)

• summary(fit)

• Same idea as binary logistic but for 3+ outcome categories. One category is the baseline; all others are compared to it.

model selection

• null <- glm(y ~ 1, data=df, family=binomial)

• full <- glm(y ~ ., data=df, family=binomial)

• step(full, direction="backward") # same stepwise process as lm()