Distribution Shift

Handling Data Drift

Models fail when data changes.

Types

Covariate shift: P(X) changes, P(Y|X) same. Concept shift: P(Y|X) changes. Prior probability shift: P(Y) changes.

Detection

Monitor input distributions. Monitor predictions. Monitor accuracy.

Mitigation

Retrain with new data. Online learning. Domain adaptation techniques.

Key Takeaways

1. Monitor for distribution shift
2. Covariate, concept, prior shift types
3. Retrain or adapt when detected