upshard

Save your progress

Everything on Upshard is free and readable without an account. Signing in saves what you have finished, your bookmarks and your notes — and nothing else.

or

We store your email, your progress, your bookmarks and your notes. Nothing else, ever. What we do with it.

Your account

Delete your account?

This removes your progress, bookmarks and notes immediately and permanently. It cannot be undone. Everything on Upshard stays free to read without an account.

Mutual information

Information theory · Formula · Not yet linked from a lesson

I(X;Y)=∑x,yp(x,y)log⁡p(x,y)p(x) p(y)I(X; Y) = \sum_{x, y} p(x, y) \log \frac{p(x, y)}{p(x)\, p(y)}

How many bits knowing one variable saves you on the other. Unlike correlation it catches any kind of dependence, not just linear ones, and it is symmetric in its two arguments.

Symbols

p(x,y)p(x, y)
The joint distribution.
p(x)p(y)p(x)p(y)
What the joint would be if the two were independent.
Esc
↑↓ navigate↵ open? shortcuts

Keyboard shortcuts

Press Esc to close

Anywhere

⌘K
Search and commands
/
Search
?
This sheet
Esc
Close

Reading a lesson

J
Next lesson
K
Previous lesson
B
Bookmark