architectures

LSTM

Long Short-Term Memory, a recurrent neural network architecture designed to learn long-range dependencies in sequential data. LSTMs use gating mechanisms to selectively remember or forget information, addressing the vanishing gradient problem of simple RNNs.

In practice

Before transformers dominated, LSTMs were the standard architecture for machine translation and speech recognition.

Neural Network
A computing system inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers. Neural networks learn patterns from data by adjusting connection weights through training, enabling them to make predictions and decisions.
Seq2Seq
Sequence-to-Sequence, an architecture that maps an input sequence to an output sequence of potentially different length. Seq2Seq models typically use an encoder to process the input and a decoder to generate the output.
Transformer
A neural network architecture introduced in the 2017 paper 'Attention Is All You Need' that processes sequential data using self-attention mechanisms instead of recurrence. Transformers enable massive parallelization during training, making them the foundation of modern LLMs and vision models.