Bayes' theorem
Flips a conditional probability around: it converts what you would expect to see given a hypothesis into how much the thing you saw should move your belief in it. Every probabilistic update in machine learning is a version of this.
- The posterior — belief in A after seeing B.
- The likelihood — how expected B was under A.
- The prior — belief in A beforehand.
- The evidence, which normalises the result.