Expected value
The long-run average of a random variable, each outcome weighted by how often it happens. Nearly every loss function in machine learning is an expectation you are approximating with a batch.
- The random variable.
- The probability of outcome x.
- The expectation operator.