Temperature sampling

upshard

p_i = \frac{\exp(z_i / T)}{\sum_{j} \exp(z_j / T)}

Divides the logits before the softmax, which rescales how peaked the distribution is. Below one it sharpens toward the top choice; above one it flattens; at zero it becomes plain greedy decoding.

Symbols

$T$: The temperature.
$z_i$: The logit for token i.
$p_i$: The resulting sampling probability.

Save your progress

Your account

Delete your account?

Temperature sampling

Keyboard shortcuts