Skip to content
AI’s

training

Adam Optimizer

Adaptive Moment Estimation, a popular optimization algorithm that maintains per-parameter learning rates adapted based on first and second moments of gradients. Adam combines the benefits of AdaGrad and RMSProp and works well across a wide range of problems.

In practice

Adam is the default optimizer in most deep learning frameworks and was used to train many of the largest language models.