PCA Deep Dive
PCA transforms data to principal components.
Mathematics
Compute covariance matrix. Find eigenvectors (principal axes) and eigenvalues (variance explained).
Project data onto top k eigenvectors. This gives k-dimensional representation.
Choosing Components
Scree plot shows elbow. Explained variance ratio cumulative. Retain enough for 95% variance.
Applications
Visualization: project to 2D. Denoising: remove small components. Compression: reduce dimensionality.
Standardize before PCA if scales differ.
Key Takeaways
- PCA finds orthogonal directions of max variance
- Choose k by explained variance
- Standardize before applying