training

Contrastive Learning

A self-supervised learning approach that trains models by pulling similar examples closer together and pushing dissimilar examples apart in embedding space. Contrastive learning creates powerful representations without labeled data.

In practice

CLIP uses contrastive learning to align image and text embeddings, learning that a photo of a cat should be near the text 'a photo of a cat.'

Embedding
A dense numerical vector representation of data (text, images, audio) in a high-dimensional space where semantically similar items are positioned close together. Embeddings capture meaning and relationships, enabling similarity search and clustering.
CLIP
Contrastive Language-Image Pre-training, a model by OpenAI that learns visual concepts from natural language descriptions. CLIP can match images and text in a shared embedding space, enabling zero-shot image classification without task-specific training data.
Self-Supervised Learning
A learning paradigm where the model generates its own labels from unlabeled data, typically by predicting missing or corrupted parts of the input. Self-supervised learning enables pre-training on massive unlabeled datasets before fine-tuning on smaller labeled datasets.