evaluation
Perplexity (metric)
A measure of how well a language model predicts a sample of text, calculated as the exponentiated average negative log-likelihood. Lower perplexity indicates the model is less surprised by the text, suggesting better language understanding.
In practice
A model with perplexity of 10 on a test set is choosing from effectively 10 equally likely next tokens on average.