architectures

BERT

Bidirectional Encoder Representations from Transformers, a model developed by Google that reads text in both directions simultaneously. BERT revolutionized NLP by introducing effective pre-training for understanding tasks like classification, question answering, and named entity recognition.

In practice

BERT powers Google Search's understanding of queries, helping it grasp the meaning of words in context.

Transformer
A neural network architecture introduced in the 2017 paper 'Attention Is All You Need' that processes sequential data using self-attention mechanisms instead of recurrence. Transformers enable massive parallelization during training, making them the foundation of modern LLMs and vision models.
NLP
Natural Language Processing, a field of AI focused on enabling computers to understand, interpret, and generate human language. NLP encompasses tasks like translation, sentiment analysis, named entity recognition, and question answering.
Encoder-Decoder
A neural network architecture with two main components: an encoder that processes and compresses the input into a representation, and a decoder that generates output from that representation. This pattern is fundamental to translation, summarization, and other transformation tasks.