Applied Machine Learning - Ch5: Logistic Regression Classifiers

What is Logistic Regression?

Logistic Regression is a classification technique that calculates probabilities for binary outcomes (0 or 1) based on input features.

How does the Logistic Regression process work?

It operates in two stages: Stage One, Calculates probabilities of belonging to each target class. Stage Two, Assigns the class with the highest probability.

What is the logit in Logistic Regression?

The logit is a function that models the outcome variable as a linear function of predictors, allowing probabilities to be mapped back to a 0/1 outcome.

What are odds, and how are they calculated?

Odds are defined as the ratio of the probability of an event occurring to the probability of it not occurring.

How do you calculate log-odds?

Log-odds are calculated by taking the natural logarithm of the odds.

How can we visualize the relationship between probability, odds, and log-odds using Python?

You can create a table that calculates odds and log-odds for various probabilities and visualize the results using a bar chart.

What are the steps involved in logistic regression?

1. Estimate the linear logit function to minimize classification error.

2. Calculate probabilities from the logit function.

3. Use a probability cut-off to predict class labels.

How can logistic regression be applied to a binary classification problem?

By using predictor variables (like hours studied and slept) to predict outcomes (such as passing or failing an exam).

How do you interpret the coefficients from a logistic regression model?

Each coefficient indicates the change in the log-odds of the dependent variable for a one-unit change in the respective independent variable.

How do you make predictions using a logistic regression model?

You input the new data into the trained model to get predicted probabilities, which are then classified based on a threshold (commonly 0.5).

What does the output of a logistic regression model include?

It includes coefficients, intercepts, predicted probabilities, and class predictions for new observations.

What is the significance of the threshold in classification?

The threshold determines the cutoff for classifying predictions as positive or negative. A common threshold is 0.5.

How do you interpret the results of a logistic regression prediction?

The log-odds show the likelihood of the event occurring; probabilities indicate the estimated chance of passing, and the predicted class reflects whether the event is likely to occur based on the threshold