architectures

Vision Transformer

A transformer architecture adapted for image processing that divides images into patches and processes them as a sequence, similar to how text transformers process tokens. ViTs have matched or exceeded CNNs on many computer vision benchmarks.

In practice

ViTs power image understanding in multimodal models like GPT-4V and Claude, enabling them to analyze uploaded photos.

Transformer
A neural network architecture introduced in the 2017 paper 'Attention Is All You Need' that processes sequential data using self-attention mechanisms instead of recurrence. Transformers enable massive parallelization during training, making them the foundation of modern LLMs and vision models.
Computer Vision
A field of AI that enables computers to interpret and understand visual information from the world, including images and videos. Computer vision tasks include object detection, image classification, facial recognition, and scene understanding.
Image Classification
A computer vision task that assigns one or more labels to an entire image based on its content. Image classification is one of the foundational tasks in computer vision and has seen dramatic improvement with deep learning.