Temporal-difference error
The gap between what you predicted and what the next step plus the next prediction say. It is the learning signal for every TD method, and it lets you learn from a single transition without waiting for the episode to end.
- The error at step t.
- The current prediction.
- The one-step bootstrapped target.