architectures

Cross-Attention

An attention mechanism where queries come from one sequence while keys and values come from a different sequence. Cross-attention enables models to relate information across different inputs, such as an image and text in multimodal models.

In practice

In image captioning, cross-attention allows the text decoder to attend to different regions of the encoded image.

Attention Mechanism
A component in neural networks that allows the model to dynamically focus on the most relevant parts of the input when producing each part of the output. Attention computes weighted sums of input representations, where weights reflect relevance.
Encoder-Decoder
A neural network architecture with two main components: an encoder that processes and compresses the input into a representation, and a decoder that generates output from that representation. This pattern is fundamental to translation, summarization, and other transformation tasks.
Multimodal AI
AI systems that can process and generate multiple types of data, such as text, images, audio, and video, within a single model. Multimodal models can understand relationships across different data types and translate between modalities.