Lecture 7: Logistic Regression

What is the goal of logistic regression?

To estimate the probability of a binary outcome using a logistic function.

How does logistic regression differ from linear regression?

It maps outputs to probabilities instead of direct values

Why is squared error not used in logistic regression?

Squared error leads to non-convex optimization and poor gradient behavior.

Why is gradient descent used in logistic regression?

Because the hypothesis function doesn’t have a closed-form solution.

What does gradient descent minimize in logistic regression?

It minimizes the cross-entropy error function

What is the role of step size η in gradient descent?

It controls how far the update moves in each step

What happens if η is too small?

Convergence is slow

What happens if η is too large?

The updates become unstable and may diverge

What are the key steps in the logistic regression algorithm?

1. Initialize w0 2. Compute the gradient. 3. Update weights. 4. Repeat until convergence.

How does logistic regression compare to perceptron learning?

It improves perceptron learning by using probabilities and cross-entropy loss