L1 regularization

Optimization · Formula · Appears in 1 lesson

Ltotal=L+λ∑i∣θi∣L_{\text{total}} = L + \lambda \sum_{i} |\theta_i|

Penalises the absolute size of the weights instead of the square. The constant gradient near zero drives weights exactly to zero rather than merely close, which is what makes it a feature selector.

Symbols

λ\lambda
The regularisation strength.
∣θi∣|\theta_i|
The absolute value of one weight.
Esc
↑↓ navigate↵ open? shortcuts

Keyboard shortcuts

Press Esc to close

Anywhere

⌘K
Search and commands
/
Search
?
This sheet
Esc
Close

Reading a lesson

J
Next lesson
K
Previous lesson
B
Bookmark