Transfer Learning
Use pre-trained models for new tasks.
Approaches
Feature extraction: freeze base, train new classifier. Fine-tuning: unfreeze some layers, train end-to-end.
Full fine-tuning: unfreeze all, lower learning rate.
Image Models
ImageNet pre-trained: VGG, ResNet, Inception, EfficientNet. Keras: applications module.
base_model.trainable = False for feature extraction. Then unfreeze for fine-tuning.
Text Models
BERT, RoBERTa, ALBERT pre-trained. Hugging Face Transformers.
Fine-tune on task-specific data. Lower learning rate than training from scratch.
Key Takeaways
- Transfer learning uses pre-trained models
- Feature extraction or fine-tuning
- Lower learning rate for fine-tuning