Logistic Regression

Logistic Regression Basics

Logistic regression models probability of binary outcome.

Sigmoid Function

σ(z) = 1/(1+e^(-z)). Maps to [0,1] probability. Decision boundary at 0.5.

Cost Function

Cross-entropy loss: -[y log(ŷ) + (1-y) log(1-ŷ)]. Convex, has global minimum.

Optimize with gradient descent or other algorithms.

Multi-class

One-vs-Rest: train binary classifier per class. Softmax: directly models multi-class probabilities.

Key Takeaways

1. Logistic regression uses sigmoid for probability
2. Cross-entropy loss is convex
3. Naturally handles multi-class with softmax