Perplexity

upshard

\mathrm{PPL} = \exp\!\left(-\frac{1}{N} \sum_{i=1}^{N} \log p(x_i \mid x_{<i})\right)

The exponential of average cross-entropy, read as an effective number of choices. A perplexity of 20 means the model is about as uncertain as if it were picking uniformly among 20 tokens at every step.

Symbols

$N$: The number of tokens scored.
$p(x_i \mid x_{<i})$: The probability the model gave the true next token.
$\exp$: Converts average log-loss back into a count of choices.

Save your progress

Your account

Delete your account?

Perplexity

Keyboard shortcuts