training
Normalization
Techniques for scaling and shifting data or intermediate representations to improve training stability and speed. Common forms include batch normalization, layer normalization, and RMS normalization, each operating across different dimensions.
In practice
Layer normalization, used in transformers, normalizes across feature dimensions within each example independently.