Multilingual NLP

Cross-Language Models

Handle multiple languages.

mBERT

Multilingual BERT. Pre-trained on many languages.

XLM-R

Cross-lingual language model. RoBERTa trained on 100 languages.

Translation

Zero-shot translation. Multilingual MT. Massive MT.

Key Takeaways

1. mBERT for multilingual
2. XLM-R large-scale
3. Cross-lingual transfer