architectures

T5

Text-to-Text Transfer Transformer, a model by Google that frames all NLP tasks as text-to-text problems. T5 converts inputs and outputs into text strings, allowing a single model architecture to handle translation, summarization, classification, and more.

In practice

T5 handles sentiment analysis by inputting 'classify: This movie was great' and outputting 'positive'.

Encoder-Decoder
A neural network architecture with two main components: an encoder that processes and compresses the input into a representation, and a decoder that generates output from that representation. This pattern is fundamental to translation, summarization, and other transformation tasks.
Transformer
A neural network architecture introduced in the 2017 paper 'Attention Is All You Need' that processes sequential data using self-attention mechanisms instead of recurrence. Transformers enable massive parallelization during training, making them the foundation of modern LLMs and vision models.
Transfer Learning
A technique where knowledge gained from training on one task is applied to a different but related task. Transfer learning enables models to achieve strong performance on new tasks with limited data by leveraging patterns learned from large-scale pre-training.