K-Nearest Neighbors Algorithm

KNN Deep Dive

Instance-based learning algorithm.

Algorithm

Find K closest training examples. Majority vote for classification. Average for regression.

Distance Metrics

Euclidean. Manhattan. Minkowski. Hamming for categorical.

Considerations

Choose K: small = complex, large = smooth. Feature scaling required. Computationally expensive at inference.

Key Takeaways

1. Lazy learning algorithm
2. K value affects complexity
3. Scale features for distance