Linear Discriminant Analysis

LDA for Classification

LDA finds linear combinations that maximize class separation.

vs PCA

PCA: unsupervised, maximizes variance. LDA: supervised, maximizes class separation.

LDA also reduces dimensionality. Up to k-1 components for k classes.

Fisher's Criterion

Maximize between-class variance / within-class variance. This creates discriminative projection.

Assumes

Normal distributions, equal covariance. Works well even when violated.

Key Takeaways

1. LDA is supervised dimensionality reduction
2. Maximizes class separability
3. Assumes normal, equal covariance