Support Vector Machines

SVM Fundamentals

SVMs find optimal hyperplane separating classes.

Maximum Margin

Support vectors define margin. Maximize margin between classes. Hard margin for linearly separable.

Soft margin (C parameter) allows misclassification for non-separable data.

Kernels

Linear kernel: original feature space. RBF kernel: infinite-dimensional space.

kernel='rbf' is default. gamma controls RBF width. C controls regularization.

Multi-class Classification

One-vs-One: trains n*(n-1)/2 classifiers. One-vs-Rest: trains n classifiers.

Key Takeaways

1. SVMs maximize margin between classes
2. RBF kernel handles non-linear boundaries
3. C and gamma are key hyperparameters