Softmax

Deep learning · Formula · Appears in 5 lessons

σ(z)i=ezi∑j=1Kezj\sigma(\mathbf{z})_i = \frac{e^{z_i}}{\sum_{j=1}^{K} e^{z_j}}

Turns a vector of unbounded scores into a probability distribution: exponentiate so everything is positive, then divide by the total so the parts sum to one. Larger gaps between scores produce a sharper distribution.

Symbols

z\mathbf{z}
The vector of raw scores, usually called logits.
KK
The number of classes.
ee
The exponential, which makes every score positive and ordering-preserving.
Esc
↑↓ navigate↵ open? shortcuts

Keyboard shortcuts

Press Esc to close

Anywhere

⌘K
Search and commands
/
Search
?
This sheet
Esc
Close

Reading a lesson

J
Next lesson
K
Previous lesson
B
Bookmark