# Feeding, Motivation and Reward

> A midbrain dopamine neuron's quick burst or pause reports a reward prediction error, whether things turned out better or worse than predicted, and hunger changes how much the reward is worth. How does the brain decide that you are hungry, and how does it learn which cues lead to food? You follow the evidence from joined-circulation mice to the hormone leptin and to two opposed groups of arcuate neurons, then read the monkey dopamine recordings and see why the burst moves from the juice to a cue that predicts it. Afterwards you can compute a prediction error with a simple learning rule, tell wanting from liking in the data, and say what a recording, a stimulation and a silencing experiment can each show.

Course: Neuroscience (Homeostasis and affect). Level: university elective. About 33 minutes.
Lesson page (link this one when citing): https://stemhop.com/lessons/motivation-and-reward/

Builds on: Autonomic Control and the Hypothalamus · Transmitter Systems and Drugs · Synaptic Plasticity

## What you'll learn

- Describe the hypothalamic feeding circuit (leptin from fat, arcuate AgRP/NPY neurons that promote feeding, POMC neurons whose alpha-MSH acts on MC4R to oppose it) and trace the evidence from parabiosis to leptin cloning and cell-specific manipulation.
- Explain reward prediction error from monkey dopamine-neuron recordings, compute it with a Rescorla-Wagner or temporal-difference rule, and explain why the burst moves to the earliest cue that predicts reward without itself being predicted.
- Distinguish wanting (incentive salience) from liking (hedonic impact) using dopamine-depletion and DAT-knockdown evidence, and relate both to addiction through incentive-sensitization theory.
- Separate what a recording shows (correlation) from what stimulation shows (sufficiency) and what silencing or ablation shows (necessity).

**Researchers put a light-sensitive switch into one small group of neurons at the base of a mouse’s brain. When they shone light on those neurons, a well-fed mouse began eating within minutes.**

Aponte, Atasoy and Sternson reported this in 2011. The neurons are the arcuate agouti-related peptide (AgRP) neurons, and the switch was a light-gated channel. That one experiment raises the two questions of this lesson. How does the brain register the need for food? And how does it learn which cues and actions lead to food, or to any reward?

## Two questions, one idea

The first question is about **worth**. A hungry animal values food more than a full one: a food loses value once an animal has eaten its fill of it. Hypothalamic circuits, including the agouti-related peptide (AgRP) neurons, are thought to help set that value from the body’s fuel state. The second question is about **learning**. An animal has to find out which sounds, places and actions come before food. Midbrain dopamine neurons carry a signal suited to that job, and it is not the signal most people expect.

Three terms first. A **prediction**, here, is how much reward the animal has learned to expect from this moment on. A **burst** is a brief jump in a neuron’s firing rate (spikes per second) above its resting rate, and a **pause** is a brief drop below it.

> **The one idea to carry through this lesson.** A midbrain dopamine neuron’s quick burst or pause reports a **reward prediction error**: whether things just turned out better or worse than the animal predicted, counting both reward that arrives and news of reward to come. At the reward itself, that is the reward received minus the reward predicted. When the same reward follows the same cue trial after trial, the prediction catches up, the error at the reward shrinks toward zero, and the burst moves to the earliest cue that predicts the reward without itself being predicted. Hunger and satiety change how much the reward is worth in the first place.

The last sentence is a modelling statement: in the rules below, the body’s state changes the size of the reward, r. Behaviour supports it, not the recordings in this lesson.

Both halves of the lesson use three kinds of experiment, and each answers a different question.

| Experiment | What it does | What it can show |
| --- | --- | --- |
| Recording | measures the cells’ activity while the animal behaves | **correlation**: the activity goes with an event |
| Stimulation | switches the cells on, here with light | **sufficiency**: imposed activity is enough to produce a behaviour |
| Silencing or ablation | switches the cells off, or kills them | **necessity**: the cells are needed for the behaviour |

## Hunger: a fuel gauge and two opposed groups of neurons

### The picture first: what a fast does

Start with what happens when an animal goes without food; the evidence for each step comes after.

1. Fat cells make a hormone, **leptin**, roughly in proportion to how much fat is stored. During a fast, leptin in the blood falls.
2. In the **arcuate nucleus**, at the base of the hypothalamus, leptin normally inhibits **agouti-related peptide (AgRP) neurons** and excites **pro-opiomelanocortin (POMC) neurons**. Falling leptin, together with other signals of energy deficit, shifts AgRP neurons toward more activity and POMC neurons toward less.
3. Their shared target, the **melanocortin-4 receptor (MC4R)**, now receives less of an activating peptide (alpha-MSH, from POMC neurons) and more of a blocking one (AgRP). Think of MC4R as a stop button: POMC neurons press it, AgRP covers it.
4. The drive to seek and eat food rises.

In an animal that responds to leptin, higher leptin pushes the other way, toward stop. High fat does not guarantee that response.

The arcuate nucleus (in the brain, a **nucleus** is a cluster of cell bodies with a shared role) sits next to the median eminence, where circulating hormones reach neurons easily; [the hypothalamus lesson](https://stemhop.com/lessons/hypothalamus-autonomic/) maps the region. Its two best-studied populations are intermingled. They act on downstream targets such as the paraventricular nucleus and work alongside other hypothalamic and brainstem neurons.

*Figure (AI-generated illustration).* Leptin from fat inhibits AgRP neurons and excites POMC neurons in the arcuate nucleus; POMC's alpha-MSH presses MC4R while AgRP peptide covers it, and the balance between them sets the drive to seek and eat food. Image: Schematic of the hunger circuit: fat cells release leptin into the blood, which inhibits AgRP neurons and excites intermingled POMC neurons in the arcuate nucleus; POMC neurons release alpha-MSH that activates MC4R on a downstream neuron while AgRP peptide blocks it, and the downstream output sets the drive to eat. An inset shows the arcuate nucleus at the base of the hypothalamus.

- **AgRP neurons** make agouti-related peptide, coexpress neuropeptide Y (NPY, another peptide that promotes feeding), and release GABA, the main inhibitory transmitter. AgRP is an MC4R antagonist.
- **POMC neurons** make pro-opiomelanocortin, which is cleaved into alpha-MSH, an MC4R agonist that reduces feeding (MC4R is a G-protein-coupled receptor; see [GPCRs and G proteins](https://stemhop.com/lessons/gpcrs-and-g-proteins/)).

MC4R matters in people too: mutations in its gene are the most common single-gene cause of severe early-onset obesity in humans (Farooqi and colleagues, 2003).

Think of leptin as a fuel gauge rather than a brake. It reports stored energy. Its strongest role is probably signalling starvation when it falls: in fasting mice, the drop in leptin helps set off the body’s neuroendocrine starvation response (Ahima and colleagues, 1996). Most people with common obesity have high leptin, not too little, so the simple replacement that worked in ob/ob mice, and in rare people born unable to make leptin, does not reliably treat common obesity.

### Evidence for a hormone: joined mice

Two mouse strains, obese (ob) and diabetes (db), were both obese and both ate too much. Douglas Coleman asked whether either lacked something carried in the blood. He used **[parabiosis](https://stemhop.com/methods/extirpation-and-replacement/)**: surgically joining two animals so they exchange some blood.

| Pair joined | What happened |
| --- | --- |
| db/db with normal | The db/db mouse kept eating. The normal partner ate little and starved (Coleman and Hummel, 1969). |
| ob/ob with normal | The ob/ob partner ate less and gained less weight than an ob/ob mouse joined to another ob/ob mouse. |
| ob/ob with db/db | The ob/ob mouse stopped eating, lost weight and eventually starved. |

**Predict.** Take two hypotheses. A strain could lack a circulating factor that limits eating, or it could make the factor but be unable to respond to it. Which strain fits which hypothesis?

> **Check your prediction.** The db/db mouse makes the factor, and a lot of it: its normal partner starved, as if strongly signalled to stop eating. Yet the db/db mouse itself kept eating, so it cannot respond. The ob/ob mouse is helped by a normal partner, so it lacks the factor but can respond to it. Joined to a db/db mouse, it receives a strong dose and stops eating. That was Coleman’s reading in his 1973 paper, and it told the gene hunters what to look for: a hormone and its receptor.

### Evidence for a hormone: leptin

Jeffrey Friedman’s laboratory found the ob gene by [positional cloning](https://stemhop.com/methods/positional-cloning/) (Zhang and colleagues, 1994; the method is in [linkage and mapping](https://stemhop.com/lessons/linkage-and-mapping/)). Its product, named leptin, is a protein of about 16 kDa made by fat and present in mouse and human plasma.

**Predict.** Halaas and colleagues gave daily injections of recombinant leptin to ob/ob mice and to db/db mice. If Coleman was right, which strain should lose weight?

> **Check your prediction.** The ob/ob mice, which lack the hormone. Leptin reduced their body weight by 30% after two weeks, and did not reduce the weight of db/db mice (Halaas and colleagues, 1995). The db gene turned out to encode the leptin receptor, which signals through the pathway taught in [JAK-STAT signalling](https://stemhop.com/lessons/jak-stat/).

### Testing the circuit: enough, needed, and when

**Stimulation asks whether activity in these cells is enough.** Aponte and colleagues (2011) put [channelrhodopsin](https://stemhop.com/methods/optogenetics/), a light-gated cation channel, into AgRP neurons and excited them with 20 Hz trains of 10 ms light pulses. This was **sufficient** to evoke feeding in well-fed mice, at least for that pattern. Prolonged POMC stimulation reduced intake, and that effect required melanocortin signalling.

**Predict.** Ay mice overexpress agouti protein, which, like AgRP, blocks melanocortin receptors, so theirs are already blocked. If AgRP neurons drove feeding only by blocking MC4R, what would stimulating them do in these mice?

> **Check your prediction.** If blocking MC4R were their only route, stimulation should add nothing, because the receptors are already blocked. In fact it still evoked feeding in Ay mice (Aponte and colleagues, 2011). So AgRP neurons do not act only by opposing alpha-MSH. They also release GABA and NPY.

**Ablation asks whether the cells are needed.** Luquet and colleagues (2005) killed AgRP neurons with [diphtheria toxin](https://stemhop.com/methods/targeted-cell-ablation/). Adult mice rapidly starved, so the cells are **necessary** for feeding in adults. Ablation in newborn mice had minimal effects on feeding, which suggests that developing circuits compensate.

**Recording adds timing.** In **[fibre photometry](https://stemhop.com/methods/calcium-imaging/)**, a fluorescent calcium indicator in the cells brightens when they are active, and an optical fibre reads that light from a population of neurons in an awake mouse.

AgRP activity, high in a hungry mouse, falls within seconds when food or a food cue appears, before eating begins (Chen and colleagues, 2015; Betley and colleagues, 2015). Signals from the gut and from fat act more slowly. Nutrients infused into the stomach inhibited AgRP neurons within minutes, scaling with calories, while leptin changed their activity over hours (Beutler and colleagues, 2017). Brief AgRP stimulation that ended before food arrived still increased later eating (Chen and colleagues, 2016). Whether AgRP activity feels unpleasant or rewarding to the mouse depends on the test used, so that question stays open.

Notice what the drop means: the hunger circuit reacts to signs of food before any is eaten. How an animal learns which signs predict reward is the rest of the lesson.

## Dopamine neurons report prediction errors

**Film transcript: The Tone and the Juice**

### Film transcript: The Tone and the Juice

A sound comes before juice, every time. Once a monkey learns that, when does a dopamine neuron fire? Wolfram Schultz's group recorded single dopamine neurons in the midbrain of monkeys. A fine electrode beside one neuron picks up its spikes, as the monkey gets drops of juice. In this picture, each row is one trial, and each tick one spike, lined up on an event. First, juice arrives with no warning. About three quarters of the recorded neurons fired a brief burst. Then a sound began to predict the juice. Once the monkeys learned that, the burst came at the sound. The juice itself, now expected, got little or no burst. Why should the burst move?

Picture one trial as a strip of moments. Waiting, then the tone, a one-second delay, then the juice. Give each moment its own prediction of the reward still to come. Before the tone, the model sets the prediction to zero. That is a simplifying choice. The tone itself comes at unpredictable times. From the tone until the juice, the prediction is high. Juice is coming. After the juice, the prediction drops back. The trial is over. The error at each step is the reward that arrived, plus the change in the prediction. At the tone, no reward arrives, but the prediction jumps up. That jump is an error, and a burst. During the delay, nothing changes.

No error. At the juice, the reward arrives, but the prediction falls by the same amount. They cancel. No burst. So unexpected good news about the future makes an error, just as reward itself does. Now the tone sounds, and the juice never comes. What happens at the moment it was due?

The prediction still falls, but no reward makes up for it. The error is negative. Firing pauses at about the time the juice was due. Hollerman and Schultz recorded it. In the model, add a light that comes at unpredictable times, always two seconds before the tone. After learning, the burst moves to the light. Now the tone is already expected, so it adds no error. The burst goes to the first cue nothing else predicted. A recording shows only that firing matches the error. Does the burst actually teach? In rats, a cue added alongside one that already predicts reward is normally learned poorly. A burst imposed with light at reward made them respond to it more.

That shows an imposed burst is enough, not that natural bursts are needed. The error is the reward plus the change in prediction. After training, it comes at the first cue nothing else predicted.

From the film

- In monkeys, about three quarters of recorded dopamine neurons burst to unexpected juice, and once a tone predicted it the burst came at the tone.
- In the model, the error is the reward plus the change in prediction, so unexpected good news about the future counts as an error.
- When an expected juice is left out, firing pauses at about the time it was due.
- A recording shows only that firing matches the error; an imposed burst in rats is enough to drive learning, not shown to be needed.

Dopamine neurons in the **ventral tegmental area (VTA)** and **substantia nigra** of the midbrain project to the **striatum** (a large region deep in the forebrain that includes the nucleus accumbens) and to frontal cortex ([transmitter systems](https://stemhop.com/lessons/transmitter-systems/) covers their release and reuptake).

### The recordings

Wolfram Schultz’s group recorded single dopamine neurons in behaving monkeys, with a fine electrode beside each neuron. Picture the record as a **raster**: each row is one trial, each tick one spike, and the rows are lined up on the moment of an event, with a histogram of all the trials underneath. A column of extra ticks is a burst. A gap is a pause. Read three records before interpreting them.

1. **Juice with no warning.** About three quarters of recorded neurons in two monkeys fired a brief burst when juice arrived.
2. **After learning.** Once the monkeys learned that a sound predicted the juice, the burst came at the sound rather than at the juice (Mirenowicz and Schultz, 1994).
3. **A reward left out, or moved.** When an expected reward was left out, firing dipped below baseline at the time it should have come. A reward at an unexpected time evoked a burst (Hollerman and Schultz, 1998).

That pattern is a **reward prediction error**: at the reward, the difference between the reward received and the reward predicted.

*Figure (AI-generated illustration).* Drawn after the monkey recordings: the dopamine burst moves from unexpected juice to the tone that predicts it, and when the expected juice is left out, firing pauses at the time it was due. Image: Three schematic spike rasters from a midbrain dopamine neuron: unexpected juice evokes a burst at the juice; after a tone has come to predict the juice, the burst comes at the tone and nothing extra happens at the juice; when the expected juice is left out, the tone still evokes a burst and firing pauses when the juice was due.

**Scope.** The rule covers the quick (sub-second) responses of many, not all, midbrain dopamine neurons, recorded in trained monkeys and rodents in tasks where reward timing is predictable. It describes a signal: a recording alone does not show that the signal teaches, and the rule says nothing about pleasure. The error at the reward shrinks to zero only when the reward is the same each time; an uncertain reward leaves some error even after learning. Striatal dopamine release can also depart from firing (Going deeper). How dopamine changes striatal synapses is in [synaptic plasticity](https://stemhop.com/lessons/synaptic-plasticity/) and [the basal ganglia](https://stemhop.com/lessons/basal-ganglia/).

### A rule for learning from errors

Picture two bars over a cup of juice. One shows the reward predicted. The other shows the reward received. The gap between them is the error. If the same juice comes on every trial, the predicted bar grows by a fixed fraction of the gap each time, so the gap shrinks. That is the **Rescorla-Wagner rule**, the simplest learning rule built on a prediction error. It has two lines: first measure the gap, then close a fixed fraction of it.

> **Equation card: How a cue’s predicted value changes after one trial: measure the error, then move the prediction part of the way toward the reward.**
>
>     δ = r − V V(next trial) = V + α × δ
>
> In words: The error, delta, is the reward received minus the value predicted. Next trial’s value is the old value plus the learning rate times the error.
>
> - δ: the prediction error: positive when the reward beats the prediction, negative when it falls short
> - r: the reward that arrives: 1 for juice, 0 for none
> - V(next trial): the cue’s value after this trial’s update
> - V: the cue’s learned value: the reward it currently predicts
> - α: the learning rate, between 0 and 1: the fraction of the gap closed on each trial
>
> Sanity check: If the prediction already matches the reward (V = r), the error is zero and V stops changing. With α = 0 nothing is ever learned; with α = 1 the value jumps all the way to the reward in one trial.
> Try one number Take α = 0.5, a round value to keep the arithmetic easy, with V = 0 and juice (r = 1) on each trial. Trial 1: δ = 1 − 0 = 1. New V = 0 + 0.5 × 1 = 0.5. Trial 2: δ = 1 − 0.5 = 0.5. New V = 0.5 + 0.5 × 0.5 = 0.75. Each trial closes half of the remaining gap, so the error halves: 1, then 0.5, then 0.25 on trial 3.

These are model units, not spikes per second.

**Worked problem.** Take α = 0.30, a reward of r = 1 unit on every trial, and V = 0 at the start.

1. Trial 1: δ = 1 − 0 = 1.00. New V = 0 + 0.30 × 1.00 = 0.30.
2. Trial 2: δ = 1 − 0.30 = 0.70. New V = 0.30 + 0.30 × 0.70 = 0.51.
3. Trial 3: δ = 1 − 0.51 = 0.49. New V = 0.51 + 0.30 × 0.49 = 0.657.
4. The error shrinks by the same factor, 1 − α = 0.7, on every trial: 1.00, 0.70, 0.49 and so on. So the error left after n trials is 0.7 multiplied by itself n times, 0.7ⁿ (see [powers](https://stemhop.com/math-toolkit/#powers)). The error is also 1 − V, so V = 1 − 0.7ⁿ. After nine trials, 0.7⁹ is about 0.040, so V = 1 − 0.040 = 0.96.

Run it the other way: after long training, with V about 0.95, omit the reward, and δ = 0 − 0.95 = −0.95, a negative error at the expected time. An unpredicted reward (V = 0) gives +1.

### From the reward to the cue

The worked problem explains one result from the monkeys: as the tone comes to predict the juice, the burst at the juice fades. It does not explain the new burst at the tone. The Rescorla-Wagner rule updates the cue’s value from trial to trial but has no time within a trial, so it cannot produce an error at the moment of the cue. For that, the prediction has to run in time *within* a trial.

Picture one trained trial as a strip of moments: waiting, then the tone, a one-second delay, then the juice. Give each moment its own prediction, V, of the reward still to come from that moment on. After training:

- **Before the tone, set V to 0.** This is a simplification. The tone comes at unpredictable times, so the model treats the waiting period as a fixed baseline and counts predictions from there. A model that also learns a value for waiting gives it some value, so the jump at the tone, and the burst, are smaller.
- **From the tone until the juice, V is 1.** Once the tone has sounded, juice is coming.
- **After the juice, V is 0 again.** The trial is over.

The **temporal-difference (TD) rule**, which Schultz, Dayan and Montague (1997) applied to these recordings, computes an error at each step from one moment to the next: the reward that arrived in that step, plus the change in the prediction.

> **Equation card: The prediction error at each step within a trial: did this step bring reward, or better or worse news about reward to come?**
>
>     δ = r + V(next moment) − V(this moment)
>
> In words: The error is the reward that arrived in this step, plus the prediction at the next moment, minus the prediction at this moment.
>
> - δ: the TD prediction error for one step
> - r: the reward that arrives during this step (1 for juice, 0 for none)
> - V(next moment): the prediction of reward still to come, made at the next moment
> - V(this moment): the prediction of reward still to come, made at this moment
>
> Sanity check: If no reward arrives and the prediction does not change, δ = 0. A rise in the prediction adds to δ just as a reward does.
> Try one number Juice arrives with no warning. Before and after it the prediction is 0, because nothing signalled it. δ = 1 + 0 − 0 = +1: a full burst at the juice, the same answer the Rescorla-Wagner rule gives for an unpredicted reward.

The full rule multiplies V(next moment) by a discount factor, up to 1, so that distant rewards can count less (Going deeper); here it is 1.

Walk along the strip.

1. **Into the tone.** On the step from waiting into the tone, no reward arrives, but the prediction jumps from 0 to 1. δ = 0 + 1 − 0 = +1: a burst, at the tone.
2. **During the delay.** No reward, and the prediction stays at 1. δ = 0 + 1 − 1 = 0: nothing.
3. **At the juice.** The juice arrives (r = 1), but the prediction falls by the same amount, from 1 to 0, because the trial is over. They cancel: δ = 1 + 0 − 1 = 0. Nothing at the juice, because it was fully predicted.

At the moment of reward, the TD error is the Rescorla-Wagner error, r − V. What step 1 adds is that unexpected good news about the future produces an error just as reward itself does. That is why, with learning, the burst moves to the **earliest cue that predicts the reward without itself being predicted**. In the model, if a light came on at unpredictable times, always two seconds before the tone, the burst would move to the light, and the tone would add no error, because by the time it sounds it is already expected. The monkey data show the first step of this, from juice to tone.

**Predict.** In a trained monkey, the tone sounds but the juice is withheld. Walk the strip. What happens to δ at the tone, and at the moment the juice should have come?

> **Check your prediction.** At the tone nothing has changed: δ = +1, a burst. At the juice time no reward arrives, yet the prediction still drops from 1 to 0, so δ = 0 + 0 − 1 = −1. The negative error comes at exactly the time the juice was due. This is the dip Hollerman and Schultz recorded. In a real neuron the dip is a brief pause in firing. A dopamine neuron fires only a few spikes per second at rest, so a pause has less room than a burst (numbers in Going deeper).

The rule makes one more prediction. Suppose the tone predicts juice on only half the trials. The prediction after the tone then averages about 0.5. In model units the error is +0.5 at the tone, +0.5 when juice comes (1 + 0 − 0.5), and −0.5 when it does not (0 + 0 − 0.5). Real bursts and pauses need not scale by the same fractions, because firing does not map one to one onto model units. In monkeys, Fiorillo, Tobler and Schultz (2003) found the same direction: responses to the cue grew with the probability of reward, and responses to the reward itself shrank.

### Watch it in the model

The simulation trains the Rescorla-Wagner rule trial by trial, so it shows the error at the reward, not the strip of moments. Watch that error shrink while the cue’s value grows. Then omit the reward to see the dip, or give the reward with no cue to see the +1 burst.

*Interactive on the lesson page:* Reward prediction error over trials — a Rescorla-Wagner sketch

### Design problem

Recordings cannot show that a dopamine burst causes learning. Design an experiment that tests whether an imposed burst is sufficient to make an animal learn about a cue it would otherwise ignore. One known effect helps, **blocking**: a new cue added alongside one that already predicts reward is learned poorly, because the reward brings no error.

> **One way to do it.** Train cue A until it fully predicts reward. Then reward the compound A plus a new cue X. δ at reward is about 0, so X is normally blocked.
>
> Use TH-Cre rats, which make the DNA-recombining enzyme Cre under the control of the gene for tyrosine hydroxylase, an enzyme of dopamine synthesis. A Cre-dependent channelrhodopsin construct injected into the VTA is then expressed mainly in dopamine neurons; that specificity has to be checked, because tyrosine hydroxylase is also made by noradrenaline cells. Deliver light at the time of reward on the A plus X trials. If the burst is a teaching error, X should now gain value: test X alone and measure approach or responding. A 2013 rat study by Steinberg and colleagues reported unblocking of this kind.
>
> Controls: rats without the opsin given the same light; opsin rats stimulated at times unpaired with reward; and the probe test of X alone, without stimulation.
>
> Limits: this shows that an imposed burst is enough in these rats under this pattern. It does not show that natural bursts are needed (that requires silencing dopamine neurons at an unexpected reward, when a natural burst would occur, and showing that learning then fails), and it says nothing about pleasure.

## Wanting is not liking

**Film transcript: Wanting Is Not Liking**

### Film transcript: Wanting Is Not Liking

dopamine the brain's pleasure signal? It is a common shortcut. Let's test it. Two things need separating. Wanting is the pull of a reward and its cues. Liking is the pleasure of the reward itself. How can you measure liking in a rat, without asking? Watch its face. Sweet tastes bring tongue protrusions. Bitter tastes bring gapes. These reactions can be scored from video. Rats whose forebrain dopamine was depleted with a neurotoxin stopped seeking food. If dopamine carried pleasure, what should their faces do when sugar is placed in the mouth?

Their faces showed normal liking reactions. The seeking was gone, but the liking reactions remained. Now the other direction. Mutant mice make little of the dopamine transporter, so extra dopamine stays outside the cells. They reached sweet food sooner in a runway. They ran no faster. They just paused less on the way. But their liking reactions to sugar were no stronger. So, in these tests, dopamine moved wanting, not liking. Recall the burst that moves to cues. It is compatible with cues gaining pull, but whether it is wanting is debated. Keep the scope. A lifelong mutation is not an acute drug, and an animal's face is not a human report. Other researchers read the same results as changes in willingness to work, or in learning.

Addictive drugs raise dopamine signalling, by different routes. Incentive sensitization theory proposes that in susceptible people, repeated use can make cue-triggered wanting grow. It can grow out of proportion to liking. It offers one explanation for why craving can outlast enjoyment. It is a theory, and other accounts exist. In these tests, dopamine moved wanting, not liking. Its quick bursts mark better than predicted, and its pauses worse than predicted. They are not a direct measure of pleasure.

From the film

- In rats, depleting forebrain dopamine stopped food seeking but left liking reactions to sugar intact.
- In mutant mice with extra extracellular dopamine, animals reached sweet food sooner without stronger liking reactions.
- So in these tests dopamine moved wanting, not liking; a lifelong mutation is not an acute drug, and a rat's face is not a human report.

A common shortcut treats dopamine as the brain’s signal for pleasure. The evidence separates two things. **Wanting**, or incentive salience, is the motivational pull of a reward and its cues. **Liking** is the hedonic impact of the reward itself (Berridge and Robinson, 1998).

Liking can be measured in animals without asking, through **taste reactivity**: tongue protrusions to sweet, gapes to bitter, scored from video.

**Predict.** Rats whose forebrain dopamine was depleted with a neurotoxin stopped seeking food. If dopamine carried pleasure, what should their faces do when sugar is placed in the mouth?

> **Check your prediction.** Their positive reactions did not weaken. The rats kept normal reactions to sugar in the mouth, though they would not seek or eat food on their own: liking intact, seeking gone.

Now the other direction. The dopamine transporter (DAT) pumps released dopamine back into the cell. Mice with DAT expression at about 10% of normal have about 70% more extracellular striatal dopamine. They reached sweet food sooner in a runway because they paused less on the way; their running speed itself did not differ. They showed no increase in liking reactions to sucrose (Peciña and colleagues, 2003).

| Manipulation | Seeking food | Liking reactions to sugar |
| --- | --- | --- |
| Dopamine down (neurotoxin, rats) | falls: no seeking or eating on their own | normal |
| Dopamine up (DAT at about 10%, mice) | rises: sweet food reached sooner, with fewer pauses | not higher |

In these tests dopamine moved wanting, not liking. Both manipulations change dopamine for hours or a lifetime, not only its quick bursts, so they show that dopamine matters for wanting, not that the burst is wanting. The burst moves to cues that predict reward, which is compatible with cues gaining pull, which is how wanting is described, but how the error signal relates to incentive salience is an open question.

**Keep the scope.** A lifelong mutation is not an acute drug, and an animal’s facial reaction is not a human report. The wanting reading is also debated: other researchers explain the same dopamine results as changes in willingness to work or in learning. In Berridge’s account, liking depends instead on small opioid and endocannabinoid hotspots in the nucleus accumbens and ventral pallidum (Berridge and Kringelbach, 2015).

## In addiction, wanting can grow while liking does not

Addictive drugs raise dopamine signalling by different routes, such as blocking dopamine transporter (DAT) or making it run in reverse, pushing dopamine out (see [transmitter systems](https://stemhop.com/lessons/transmitter-systems/)). **Incentive-sensitization theory** (Robinson and Berridge) proposes that in susceptible individuals, repeated drug use makes the circuits that assign incentive salience hypersensitive, so drug wanting, especially when triggered by cues, can grow out of proportion to drug liking. It offers an explanation for why craving can persist after the drug has stopped being enjoyable.

It is a theory, and addiction is not only dopamine. Other accounts stress habit learning in the dorsal striatum, weakened prefrontal control, and withdrawal and stress systems, and genes, environment and drug history all shape who becomes addicted. Forward, [attention and decisions](https://stemhop.com/lessons/attention-and-decisions/) shows how learned values guide choice.

## Going deeper

### Feeding centres before the hormone

For decades feeding was explained by two “centres”. In rats, Hetherington and Ranson (1940) found that lesions of the ventromedial hypothalamus produced overeating and obesity, and Anand and Brobeck (1951) found that lesions of the lateral hypothalamus stopped animals eating. These early lesions destroyed everything in one place, including fibres passing through, so the labels described where damage changed feeding, not what the cells there compute. The idea of single centres has been superseded, although both regions still contain neurons that matter for feeding and body weight.

### How leptin acts on POMC neurons, and on others

In recordings from identified pro-opiomelanocortin (POMC) neurons in mouse brain slices, leptin depolarized the cells through a nonspecific cation conductance and reduced their inhibitory input from local NPY/GABA neurons (Cowley and colleagues, 2001). Neither population carries leptin’s effect alone: deleting leptin receptors from POMC neurons causes only mild obesity, and deleting them from both agouti-related peptide (AgRP) and POMC neurons still leaves part of the response, so other leptin-sensitive neurons share the work (Balthasar and colleagues, 2004; van de Wall and colleagues, 2008).

### The full temporal-difference rule

With a discount factor γ (“gamma”), a number from 0 to 1 chosen by the modeller, the TD error at time t is

> **δ(t) = r(t) + γ × V(t+1) − V(t)**

Here t counts time steps, and V(t+1) is the prediction one step later. With γ = 1 this is the rule in the lesson; with γ = 0 only the reward in the current step counts. At the end of a trial V(t+1) = 0, so the error at the reward is still r − V. Discounting shrinks the cue burst when the reward is far off, because each step of delay multiplies the reward’s worth by γ once more. In the model, with γ = 0.95 per time step and ten time steps between cue and reward, the burst at the cue is 0.95 multiplied by itself ten times, 0.95¹⁰, about 0.60 of a full burst. So the model predicts smaller cue responses for longer cue-reward delays.

### An error that travels backward

During learning, the simplest TD model, with one state for each moment of the delay, also predicts small errors at moments between the cue and the reward. Over trials, the error appears to travel backward from the reward toward the cue. Early monkey recordings did not clearly show this. A 2022 mouse study (Amo and colleagues) reported a gradual backward shift of dopamine responses during learning. How general it is depends on the task and on how the brain represents time, so treat it as an open question, not a settled failure or success of the model.

### Why a pause has less room than a burst

Take illustrative numbers: a resting rate of 5 spikes per second, and a gain of 20 spikes per second for each unit of error. In this mapping, rate = 5 + 20 × δ. A δ of +1 gives 25 spikes per second. A δ of −0.5 would need 5 − 10 = −5 spikes per second, and a δ of −0.95 would need −14. A firing rate cannot go below zero. Setting 5 + 20 × δ = 0 gives δ = −5 ÷ 20 = −0.25, so every error of −0.25 or below gives the same zero firing: the pause saturates while a burst of the same size keeps growing. The floor limits how deep a pause can be; it does not by itself set how long the pause lasts. In monkey recordings, Bayer and Glimcher (2005) found that positive errors were coded roughly in proportion to firing rate, while negative errors had a compressed range.

### Release is not the same as firing

In rats, dopamine release in the nucleus accumbens tracked reward expectation without matching changes in ventral tegmental area (VTA) firing (Mohebi and colleagues, 2019). How local control of release combines with spike timing to set what striatal synapses learn is unresolved.

---

From STEMhop (https://stemhop.com), free lessons by Christopher Deppmann. Lesson text © the authors. Practice questions: https://stemhop.com/lessons/motivation-and-reward/
