515 Logistic Regression: A Comprehensive Guide

Logistic Regression

A statistical method used for predictive analytics, particularly for categorical response variables.

Categorical Response Variable

An outcome or response variable that falls into distinct categories, usually binary or binomial.

Probability-Based

Logistic regression models the probability of an event occurring, with outcomes between 0 and 1.

Logit Transformation

A method that converts the odds of an outcome to a continuous criterion using the natural logarithm.

Logistic Function

An S-shaped function that predicts the probability of an outcome, ranging between 0 and 1.

Maximum Likelihood Estimation (MLE)

The method used to estimate coefficients in logistic regression by maximizing the likelihood function.

Similarities with Linear Regression

Both aim to model the relationship between response and explanatory variables using past observations.

Differences with Linear Regression

Logistic regression is designed for categorical response variables, whereas linear regression is for continuous numerical variables.

Model Development

Involves data consisting of observations with attributes and a categorical response variable.

Odds Ratio

A measure used in logistic regression to compare the odds of an event occurring in different groups.

Likelihood-Ratio Test

A test used to compare nested logistic regression models by their log-likelihoods.

Misclassification Rate

The proportion of incorrect classifications made by a classifier.

Confusion Matrix

A tool to evaluate performance showing counts of true positives, true negatives, false positives, and false negatives.

Sensitivity and Specificity

Sensitivity measures the true positive rate, while specificity measures the true negative rate.

Classification Threshold

A common decision rule in logistic regression where a probability greater than 0.5 classifies an observation as 1.

Limitations of Logistic Regression

Assumes a linear relationship between predictors and log-odds, sensitive to outliers, and does not establish causality.