What does “Perplexity” mean?
A language-model metric measuring how surprised the model is by true next tokens.
How Perplexity Works
A language model reads a held-out news article and rates how likely each actual next word was. If it kept assigning low probability to the real words, perplexity is high, which signals a weaker fit to that kind of text.
Practical Example
A language model reads a held-out news article and rates how likely each actual next word was. If it kept assigning low probability to the real words, perplexity is high, which signals a weaker fit to that kind of text.
Why It Matters
Perplexity is a standard way to compare language models on the same text. It does not by itself measure helpfulness or truthfulness.
Common Questions About Perplexity
Q: Is lower perplexity better?
Yes, for comparison on the same data and tokenization. It means the model assigned higher probability to the actual text.
Q: Can I compare perplexity across models?
Only fairly if they use the same tokenizer and test text.
Q: Does low perplexity mean accurate answers?
No. It measures fit to text, not whether statements are correct or useful.