Sequence-to-sequence models: T5 and BART

Sequence-to-sequence models: T5 and BART

Key points

• Seq2seq models have an encoder that reads input and a decoder that writes output.
• T5 frames every task as text-to-text, using a prefix like "summarize:".
• BART is pre-trained by reconstructing corrupted text.