Continual Learning

Learning Continuously

Learn from streaming data without forgetting.

Challenges

Catastrophic forgetting: old tasks forgotten. Stability-plasticity dilemma.

Approaches

Regularization: penalize changes to important weights. Replay: store or generate old examples. Progressive networks: add new columns.

Evaluation

Forward transfer: apply to new task. Backward transfer: improve old tasks.

Key Takeaways

1. Avoid catastrophic forgetting
2. Regularization, replay, progressive networks
3. Track forward/backward transfer