Temporal difference reinforcement learning codes for surprising outcomes by comparing expectations to actual results.

“Um uh it's called temporal difference reinforcement learning. Um is the fact that in the world these expectations are going through their own trajectory. All right. And that's what dopamine is coding for. Any learning rule should code for the surprising outcome.”