The Tone and the Juice
Read the transcript of “The Tone and the Juice”
Film transcript: The Tone and the Juice
A sound comes before juice, every time. Once a monkey learns that, when does a dopamine neuron fire? Wolfram Schultz's group recorded single dopamine neurons in the midbrain of monkeys. A fine electrode beside one neuron picks up its spikes, as the monkey gets drops of juice. In this picture, each row is one trial, and each tick one spike, lined up on an event. First, juice arrives with no warning. About three quarters of the recorded neurons fired a brief burst. Then a sound began to predict the juice. Once the monkeys learned that, the burst came at the sound. The juice itself, now expected, got little or no burst. Why should the burst move?
Picture one trial as a strip of moments. Waiting, then the tone, a one-second delay, then the juice. Give each moment its own prediction of the reward still to come. Before the tone, the model sets the prediction to zero. That is a simplifying choice. The tone itself comes at unpredictable times. From the tone until the juice, the prediction is high. Juice is coming. After the juice, the prediction drops back. The trial is over. The error at each step is the reward that arrived, plus the change in the prediction. At the tone, no reward arrives, but the prediction jumps up. That jump is an error, and a burst. During the delay, nothing changes.
No error. At the juice, the reward arrives, but the prediction falls by the same amount. They cancel. No burst. So unexpected good news about the future makes an error, just as reward itself does. Now the tone sounds, and the juice never comes. What happens at the moment it was due?
The prediction still falls, but no reward makes up for it. The error is negative. Firing pauses at about the time the juice was due. Hollerman and Schultz recorded it. In the model, add a light that comes at unpredictable times, always two seconds before the tone. After learning, the burst moves to the light. Now the tone is already expected, so it adds no error. The burst goes to the first cue nothing else predicted. A recording shows only that firing matches the error. Does the burst actually teach? In rats, a cue added alongside one that already predicts reward is normally learned poorly. A burst imposed with light at reward made them respond to it more.
That shows an imposed burst is enough, not that natural bursts are needed. The error is the reward plus the change in prediction. After training, it comes at the first cue nothing else predicted.
A well-fed mouse starts eating within minutes when one small group of hypothalamic neurons is switched on with light. What are those neurons for?
Continue in the lesson: Feeding, Motivation and Reward →
3:10 · Free, with captions and a transcript. The lesson has the full text, the questions and the sources.
More films from this lesson
How this film was made. Narration: an AI narrator voice (Kokoro, af_heart) reading narration written from this lesson's text. Animation: AI-generated in code (Manim) by Claude, a schematic with no drawn molecular structures. Captions come from the narration script. See something wrong? Report it.