Sequence-to-Sequence Models

Seq2Seq Architecture

Map input sequence to output sequence.

Encoder-Decoder

Encoder processes input sequence, produces context vector. Decoder generates output, conditioned on context.

LSTM/GRU for both. Context passed to each decoder step.

Attention

Attention allows decoder to look at all encoder states. Improves long sequences.

Bahdanau attention: alignment scores, weighted context. Multi-head attention.

Applications

Machine translation. Text summarization. Question answering. Chatbots.

Key Takeaways

1. Encoder-decoder is core seq2seq
2. Attention improves long sequences
3. Foundation for translation, summarization