Q-learning update

upshard

Q(s, a) \leftarrow Q(s, a) + \alpha\!\left[r + \gamma \max_{a'} Q(s', a') - Q(s, a)\right]

Nudges the value of the action you took toward the reward plus the best value available next. The max is over actions you did not necessarily take, which is what makes it off-policy and lets it learn the optimal policy while exploring.

Symbols

$\alpha$: The learning rate.
$\max_{a'} Q(s', a')$: The best value available in the next state.
$r$: The reward just received.

Save your progress

Your account

Delete your account?

Q-learning update

Keyboard shortcuts