# Combining Senses and Predicting Them

> A signal is worth what it tells you about the world: the more reliable of two senses counts for more, and some pathways subtract the predictable result of your own movement. A nervous system has to decide how much to trust each sense, and how much of what its receptors report is only the consequence of its own movement; in monkey vestibular neurons the subtraction ran only while the feedback matched. You predict where a combined estimate lands and how blur reverses it, see the reliability rule in people and in monkey MSTd, and keep collicular enhancement apart as a single-neuron measure. Then receptors that report the animal's own movement, a fish that learns a negative image of its own pulse, monkey vestibular neurons whose cancellation fails on a mismatch, the self-tickle experiments and mouse cortical movement and mismatch signals. Going deeper holds causal inference, correlated cues, behavioural state and looming. Afterwards you can compute a cue weight from two variances, read an enhancement figure, and design the control that separates peripheral from central cancellation.

Course: Sensory Systems (Integration and repair). Level: advanced university and graduate. About 33 minutes.
Lesson page (link this one when citing): https://stemhop.com/lessons/multisensory-and-prediction/

Builds on: Meissner and Pacinian Corpuscles: Flutter and Vibration · Balance: Two Codes for Every Head Movement · Five Ways to Turn a Stimulus into a Current

## What you'll learn

- Calculate the weights, combined estimate and combined variance for two independent Gaussian cues, and state what agreement with behaviour does and does not show.
- Explain percentage enhancement and inverse effectiveness in superior-colliculus recordings, and the spatial and temporal conditions under which enhancement was maximal.
- Describe how a copy of a motor command can be turned into a cancelling prediction in a fish, and why the vestibular cancellation in monkeys is conditional on matching proprioceptive feedback.
- Explain why reduced or altered responses during movement do not by themselves show predictive cancellation, and design the comparison that would test whether self-generated vibration is attenuated centrally, given that the peripheral afferents respond to it.

**Your eyes and your hand disagree about the height of a bar. Which one do you believe? And why is it so hard to tickle yourself, when a stranger’s finger on the same spot makes you squirm?**

Both questions ask what a signal is worth. When two senses report the same event, a nervous system has to decide how much to trust each one. When the animal moves, its receptors report that movement along with the world, and the nervous system has to decide how much of that input to discount. The full answer to the tickle question is not settled, but it belongs to the second problem. This lesson starts where [sensory coding](https://stemhop.com/lessons/sensory-coding/) stops: the spikes are already a code, and the question is what is done with several streams of them.

## Count each sense by its reliability

### Two wobbly rulers, one bar

Suppose you judge the height of a bar by looking at it and by gripping it. Each sense is like a ruler that wobbles: repeat the judgement and the answers scatter around the true height. The size of that scatter is the standard deviation (SD), and the SD squared is the **variance**. A sense with a small SD is **reliable**.

Take illustrative numbers. Vision has an SD of 2 mm and touch an SD of 4 mm. On one trial vision says 50 mm and touch says 54 mm.

**Predict.** You must give one answer. Should it land halfway, at 52 mm, or nearer one sense? And should the combined answer be more or less precise than vision alone?

> **Check your prediction.** It lands much nearer vision, at 50.8 mm, and it is more precise than either sense alone.
>
> The rule behind it has three conditions. Each sense is right on average (its estimate is **unbiased**). The errors of the two senses are unrelated (**independent**). You had no expectation of the height before you looked (a **uniform prior**). Then give each sense a weight equal to its **precision**, one over its variance, divided by the sum of the precisions:
>
> > **wV = (1/σV²) / (1/σV² + 1/σH²), wH = 1 − wV** **combined estimate = wV × estimate\_V + wH × estimate\_H** **combined variance = σV² σH² / (σV² + σH²)**
>
> Worked through:
>
> 1. Variances: 4 mm² and 16 mm². Precisions: 1/4 and 1/16 per mm².
> 2. Weights: wV = (1/4)/(1/4 + 1/16) = 0.8 and wH = 0.2.
> 3. Combined estimate: 0.8 × 50 + 0.2 × 54 = 50.8 mm, which is vision moved a fraction wH of the way toward touch, 50 + 0.2 × (54 − 50).
> 4. Combined variance: (4 × 16)/(4 + 16) = 3.2 mm², so SD = 1.789 mm. Check: 0.8² × 4 + 0.2² × 16 = 2.56 + 0.64 = 3.2 mm².
>
> The halfway answer is the trap. An equal-weight average has variance 0.5² × 4 + 0.5² × 16 = 5 mm², worse even than vision alone at 4 mm². Weighting by reliability is what lets the weaker sense help.
>
> If the errors are independent, these weights give the least variance of any unbiased weighted average. With Gaussian (bell-shaped) errors, this combination is also the **maximum-likelihood** estimate, the height that makes both readings most probable, which is the model tested in people below. With correlated errors the best weights must take the correlation into account (Going deeper).

**Predict.** Now blur the picture until vision has an SD of 8 mm. Touch stays at 4 mm, and the two readings stay at 50 and 54 mm. Where does the combined answer go?

> **Check your prediction.** It moves most of the way to touch. The variance of vision is now 64 mm², so wV = (1/64)/(1/64 + 1/16) = 0.2 and wH = 0.8. The estimate is 0.2 × 50 + 0.8 × 54 = 53.2 mm, and the variance is (64 × 16)/(64 + 16) = 12.8 mm², an SD of 3.58 mm. This is the blur reversal in miniature.

Alais and Burr (Current Biology, 2004) found the reversal for sound and sight in human localization. Vision usually dominates where a sound seems to come from, but when they blurred the visual stimulus severely, sound captured the perceived location of the visual one. So “vision wins” is a statement about typical reliability, not a law.

*Interactive on the lesson page:* Weighing two cues by reliability

### The test in people

Ernst and Banks (Nature, 2002) tested this with four human observers who judged bar height by sight and by touch. A maximum-likelihood model, built from each observer’s single-sense performance, closely resembled the combined judgements, including when noise made vision less reliable (figure). Read the scope: one task, four people, a behavioural fit. Agreement supports the model as a description of what observers do; it does not locate the weighting in the brain or show that every pair of cues is combined this way.

*Figure (AI-generated illustration).* Schematic of the Ernst and Banks bar-height task with four observers. In the real experiment vision came from a random-dot stereogram and touch from force-feedback surfaces; adding noise to the visual dot depths made vision less reliable. Combined judgements were compared with predictions from each observer's single-sense performance. The drawing is simplified. Image: three simplified panels of a person judging the height of an object: by sight only, by touch only, and by both together, with visual noise shown in the last panel. The drawing is a schematic of the experiment, not its actual apparatus.

> **The one idea to carry through this lesson.** A signal is worth what it tells you about the world. When two senses report the same event, the more reliable one counts for more. When part of the input is the predictable result of your own movement, it tells you little that is new about the outside world, and some pathways subtract a prediction of it so that what the world adds stands out. In the monkey vestibular neurons tested, that subtraction ran only while the body’s feedback matched the prediction.

The idea covers the human and monkey cue-combination results and, below, the fish and the monkey vestibular neurons. It assumes one event: the weighting applies when both signals are treated as coming from the same cause (Going deeper). It does not explain inverse effectiveness in collicular neurons, which describes the gain of single neurons, or the general change in cortical responses with running, which is not tied to a prediction. It does not say that all self-caused input is thrown away: gripping the bar is your own movement, and the touch it produces is how you measure the bar. Nor does it say where subtraction happens: in monkey vestibular nuclei and the fish’s electrosensory lobe it is central. In larval zebrafish, efferent neurons inhibit lateral-line **afferents** (the sensory fibres carrying signals in from the lateral line, a row of water-flow sensors along the body) during swimming, a damping at the periphery. And the match condition belongs to the monkey: in the fish, input the prediction did not anticipate is simply left over after the subtraction.

### Where is the weighting?

Behaviour alone cannot say where the weighting happens; the closest neural evidence comes from monkeys judging heading, the direction of their own motion, from visual and vestibular cues. Area MSTd, in monkey visual cortex, has neurons that respond to both kinds of motion cue. When the visual motion was made less reliable, the monkeys leaned more on the vestibular cue, and the activity decoded from MSTd neurons shifted its weights the same way (Fetsch and colleagues, Nature Neuroscience, 2012). That links the reliability rule to a population of neurons, without showing that this area is where the rule is computed.

### A single-neuron measure that is not the reliability rule

The superior colliculus, a midbrain structure that receives visual, auditory and somatosensory input, gives a result about single neurons rather than observers. In 1983 Meredith and Stein reported in Science that a cue of another sense could change the response of a superior-colliculus cell to a stimulus, and that in some conditions the response was depressed instead of enhanced. The finding was interaction, not mere convergence of inputs.

In cats, Meredith and Stein (Journal of Neurophysiology, 1986) compared the response to both cues together with the response to each alone. Percentage enhancement is the gain relative to the larger single-cue response:

> **percentage enhancement = (combined − best single-cue response) / best single-cue response × 100**

Take made-up data. A neuron fires 2 spikes to a light, 1 to a sound and 6 to both: enhancement is (6 − 2)/2 = 200%, an absolute gain of 4 spikes. A second neuron fires 20 to the light, 10 to the sound and 30 to both: enhancement is (30 − 20)/20 = 50%, an absolute gain of 10 spikes. Percentage enhancement was generally largest when the single-cue responses were weakest, which is what **inverse effectiveness** means. Note the metric: the weak neuron gains more in percent and less in spikes. Inverse effectiveness is best read as a tendency that depends on how it is measured (Going deeper).

Do not read it as the reliability rule inside one neuron. Take an optimal detector with two independent cues, each with Gaussian noise of the same spread whether or not a signal is present, and measure how detectable each cue is by its **d′** (the shift in mean response a signal causes, divided by the noise SD). The combined d′ is √(d′1² + d′2²). If both cues’ d′ grow by the same factor, the combined d′ stays the same multiple of the better cue’s at every strength: √2 for equal cues, √5/2 ≈ 1.12 for a 2 to 1 pair. So the reliability rule by itself does not predict inverse effectiveness in neuronal firing.

Two more rules came from the same laboratory. Effects depended on where the stimuli fell relative to the neuron’s receptive fields (the regions of space where a stimulus of each sense drives the cell). And in the 1987 study enhancement was greatest when the peak discharge periods of the two responses overlapped, which need not coincide with simultaneous stimulus onsets, because responses to different senses start with different delays.

Whether these single-neuron effects implement the observer-level weighting is a question these papers do not settle.

## Subtract what you caused

Now the second half of the one idea. Before any prediction can be subtracted, a nervous system has to face how much of its input it causes.

### The receptors report your own movement

Sensory input that results from the animal’s own action is called **reafference**. Input from events outside is **exafference**.

Turecek and Ginty (Neuron, 2024) asked what touch receptors signal during natural behaviour. They recorded single hindlimb units, which the authors identified as Pacinian, in freely moving mice. The units fired to the mouse’s own movements, with brief instantaneous rates up to 900 Hz (an interval of about 1.1 ms between spikes within such a burst, not an average rate). Some fired during limb movement while the mouse was suspended without limb contact with outside surfaces. External vibration transmitted along a branch could activate a sensitive unit from a distance. Tuning and sensitivity differed from unit to unit. The study did not test self-touch or tickle, and Pacinian afferents are not known to carry the tickle signal. For the end organ itself, see [vibration corpuscles](https://stemhop.com/lessons/vibration-corpuscles/).

*Figure (AI-generated illustration).* Turecek and Ginty recorded single units identified as Pacinian from freely moving mice. Units responded to the mouse's own movements; five of nine tested units also fired during limb movement while the mouse was suspended with its limbs touching no outside surface. A sensitive unit responded to vibration carried along a branch from 2.5 m, the furthest distance tested, not a measured limit. The conditions are schematic and not to scale. Image: a freely moving mouse with a recording tether, plus two schematic insets: the mouse suspended with its paws touching nothing, and a mouse on a long branch with a vibration source up to 2.5 metres away, the furthest distance tested. The drawing is not to scale and does not show the true location of the recorded endings.

The inner ear tells the same story. In alert macaques, semicircular-canal afferents encoded active and passive head-on-body rotation (the head turning on the neck) similarly (Cullen and Minor, 2002). The inner-ear basics are in [hearing and balance](https://stemhop.com/lessons/hearing-and-balance/) and [the vestibular system](https://stemhop.com/lessons/vestibular-system/).

So in these two systems the receptors pass self-generated input on to the brain. In the monkey, comparing active and passive turns places the suppression that was found beyond the receptors. The mouse study did not test whether the input is damped further along, or compare self-generated vibration with matched external vibration (see the Design problem).

## A fish that learns to cancel its own pulse

**Film transcript: A Fish That Learns to Cancel Its Own Pulse**

### Film transcript: A Fish That Learns to Cancel Its Own Pulse

An electric fish senses the world with its own pulses. Why does every pulse not mask what it listens for? Each pulse also stimulates its passive electroreceptors. These only listen, for weak fields from outside sources. The command that fires the electric organ is also copied to the electrosensory lobe. Bell found that the effect of that copy is not fixed. It changed to match the input from the receptors. Now the key test. Paralyse the fish. The brain still issues the command, but the electric organ no longer fires. Pair each command with an artificial stimulus that mimics a pulse. Then switch the stimulus off. Predict. What does the command alone now do in the lobe?

It evokes a mirror of the paired input. Where the stimulus excited the cells, the command alone inhibits them. This mirror is called a negative image. Added to the real input, it cancels it. It was learned, because it followed whatever stimulus had been paired.

How does one command become a mirror with the right timing? Kennedy and colleagues recorded in awake, paralysed fish. Granule cells respond to the command at different delays. Unipolar brush cells add delayed responses. Together they form a temporal basis, a bank of signals each peaking at a different moment. Two rules set their strengths. An input followed shortly by a broad spike in the cell's dendrites is weakened. And every active input is strengthened a little, whether or not the cell fires. On average, each active synapse weakens or strengthens according to the broad teaching spikes that follow it. With enough differently timed inputs, learning shrinks the predicted response.

So learning shrinks the response to the pulse. The command alone now dips where the pulse excited the cell. A model built on these recordings learned to cancel the predicted input. The limit: a paralysed fish, plus a model. Take this with you. Using a copy of the command, the circuit learns a timed negative image of the expected input.

From the film

- In the electric fish, a copy of the motor command is used to learn a timed negative image of the expected input, which cancels it.
- The negative image was learned, because it followed whatever stimulus had been paired with the command.
- The evidence comes from paralysed fish plus a model built on recordings.

A mormyrid electric fish senses its surroundings with electricity. It emits a pulse from its electric organ, and each pulse also stimulates its own **passive electroreceptors**, the ones that detect weak fields from outside sources. That self-generated input is what gets cancelled. If nothing were done, every pulse could mask the signals the fish is listening for.

Now the wiring. The command that fires the electric organ is also copied to the central electrosensory pathway, whose first station is the **electrosensory lobe**. A copy of a motor command sent to sensory circuits is called an **efference copy** or **corollary discharge**. Bell (Science, 1981) found that the effect of that copy is not fixed: it changed to match variations in the afferent input, and he proposed that it probably reduces the central effects of the fish’s own discharge.

**Predict.** Follow Bell’s 1981 design. Paralyse the fish with a neuromuscular blocker. The brain still issues the command, but the electric organ, which is modified muscle, no longer fires. Now pair each command with an artificial stimulus that mimics what a pulse would do to the receptors. Then switch the stimulus off and record the response in the lobe to the command alone. What do you expect to see?

> **Check your prediction.** The command alone now evokes a response that mirrors the paired input. Where the stimulus had excited the cells, the command alone inhibits them, and the reverse. This mirror is called a **negative image**: a response that mirrors the expected input and, added to it, cancels it. The image was learned, because it followed whatever stimulus had been paired with the command. In the lobe, the medium ganglion cells form these negative images.

How can a single command become a mirror with the right timing? Kennedy and colleagues (Nature Neuroscience, 2014) recorded the command response alone in awake, paralysed *Gnathonemus petersii*. **Granule cells** respond to the command at different delays, and **unipolar brush cells** contribute delayed responses. Together they give the medium ganglion cells a **temporal basis**: a set of signals, each peaking at a different moment after the command.

Learning sets the strength of each signal in the set, by two rules:

1. **Depression timed to the cell’s spikes.** A synapse is weakened when its input is followed shortly by a broad spike in the medium ganglion cell’s dendrites; the cell’s ordinary narrow output spikes do not act as this teaching signal. This is **anti-Hebbian timing-dependent depression**, the opposite of “fire together, wire together”. Here “depression” means a weaker synapse, unrelated to the depressed collicular responses above.
2. **A small boost for every input.** Each presynaptic spike strengthens its synapse a little, whether or not the cell fires (**non-associative potentiation**).

On average, a synapse weakens when broad dendritic spikes follow its input often enough within the plasticity window for depression to outweigh the boost, and strengthens when they follow too rarely. Where the pulse drives the cell harder, broad spikes follow the inputs active at that moment more often, so those inputs weaken. With enough differently timed inputs, learning shrinks the part of the response that the command predicts.

A simplified illustration: three independent time bins after the command, a linear response in arbitrary units, and enough adjustable excitatory drive in each bin. Before learning, the command drives 10 in each bin. The pulse adds 6 in the middle bin, so the cell runs at 10, 16 and 10. Learning weakens the middle input until the command alone drives only 4 there. Now, with the pulse, the cell runs at 10, 10 and 10: flat. The command alone gives 10, 4 and 10, a change of 0, −6 and 0 from before learning, a dip exactly where the pulse excited the cell. Every synapse is still excitatory; the negative image is the change from the cell’s response before learning. Where the pulse inhibits the cell, inputs strengthen by the same logic, which gives the mirror its raised parts. These are illustrative units, not measured firing rates: the real model uses overlapping, filtered signals, and learning reduces the predictable modulation rather than guaranteeing a flat line at every moment. In a model built on these recordings, this pair of rules learned to cancel the predicted input.

The limit: a paralysed fish, plus a model; it is not a recording from a swimming fish. For timing-dependent rules in general, see [synaptic plasticity](https://stemhop.com/lessons/synaptic-plasticity/), and for the cerebellar relatives of this circuit, [the cerebellum](https://stemhop.com/lessons/cerebellum/).

## Only when the feedback matches: the monkey

**Film transcript: Only When the Feedback Matches**

### Film transcript: Only When the Feedback Matches

When you turn your head, your inner ear reports the turn. Does your brain treat it like a turn someone gave you? Start with the receptors. In alert monkeys, canal afferents report active and passive turns alike. In freely moving mice, Pacinian touch afferents also fire strongly to the animal's own movements. So the receptors pass self-caused input to the brain. Whether it is damped further along, that mouse study did not test. Back to the monkey. Central vestibular-only neurons respond to head motion, not to eye movement. An active turn barely changes their firing. The same turn applied passively changes it strongly. Neck proprioceptors report how the head moves on the body.

The cancellation needed them to match the command's prediction. Brooks and Cullen then added passive motion during an active turn, activating the same proprioceptors. Predict. With the feedback no longer matching, what do the central neurons report?

They report the whole movement, active and passive parts together. The cancellation fails. So in these neurons, cancellation is central, and it holds only while the feedback matches. People show a related pattern. Self-produced touch is rated less ticklish than the same touch from outside. In one study, people moved a robot arm with one hand. A second robot touched their other palm. Predict. Add a delay between the movement and the touch. What happens to the tickle?

It feels more ticklish with a delay. Rotating the hand's movement against the touch's direction does the same. A forward model, predicting a command's sensory result, is one candidate explanation, not a settled one. Mice show a related signal. Some visual cortex neurons respond when the visual flow fails to match running. One reading: a failed prediction. Back to the one idea. Some pathways subtract a prediction of what your own movement causes. Take this with you. In these monkey neurons, the subtraction ran only while the feedback matched the prediction.

From the film

- In alert monkeys, canal afferents report active and passive turns alike, so the receptors pass self-caused input on.
- In the monkey central vestibular-only neurons tested, the subtraction of active turns ran only while neck feedback matched the prediction.
- Self-produced touch is rated less ticklish than the same touch from outside, and a forward model is one candidate explanation, not a settled one.

The fish shows how a prediction can be learned. The monkey shows that using it has a condition attached.

Two terms first. **Vestibular-only neurons** are central neurons that respond to head motion but not to eye movement. **Neck proprioceptors** are sensors in the neck muscles that report how the head is moving on the body; their signal is the proprioceptive feedback. These neurons fire all the time, and head motion **modulates** that firing, up or down depending on the direction each neuron prefers. In alert macaques, an active turn barely modulates vestibular-only neurons, while the same turn applied passively modulates them strongly, although the canal afferents report both alike (above). Roy and Cullen (2004) found that this cancellation occurred only when neck proprioceptive activation matched what the motor command predicted.

Brooks and Cullen (Journal of Neurophysiology, 2014) recorded in three awake rhesus monkeys. They found cancellation for matched self-generated motion produced by the axial muscles too. Then they combined active and passive motion so that both activated the same proprioceptors at once:

| Condition | Canal afferents | Vestibular-only neurons |
| --- | --- | --- |
| Active turn, feedback matches the prediction | encode the turn | weakly modulated by the turn |
| The same turn applied passively | encode the turn | strongly modulated by the turn |
| Active turn plus added passive motion, both activating the same proprioceptors | expected to encode the total motion (not recorded in this condition) | ? |

**Predict.** In the third row the motor command still predicts the active part, but the added passive motion changes the proprioceptive signal, so it no longer matches what the command alone predicts. What do the vestibular-only neurons report?

> **Check your prediction.** They robustly encoded the total vestibular input, active and passive parts together. In these neurons cancellation is central, and it fails without a match. The firing of these neurons is not a direct measure of conscious self-motion perception.

## What a person feels: the tickle

The observation that you can hardly tickle yourself is old; Weiskrantz, Elliott and Darlington published preliminary observations on tickling oneself in Nature in 1971. In a functional imaging study of adult volunteers, Blakemore, Wolpert and Frith (Nature Neuroscience, 1998) found that self-produced tactile stimulation was rated less ticklish than the same stimulation applied externally, with less activity in somatosensory cortex. The authors suggested that the cerebellum predicts the sensory consequences of the movement.

The matching condition can be tested in people too. Blakemore, Frith and Wolpert (Journal of Cognitive Neuroscience, 1999) used a robotic setup. Participants moved a robot arm with the left hand, and a second robot produced a touch on the right palm. Because the robots linked the two, the experimenters could change how well the touch corresponded to the movement.

**Predict.** Insert a delay between the left hand’s movement and the touch on the right palm, or rotate the direction of the left hand’s movement relative to the direction of the touch, which still runs straight across the palm. What happens to how ticklish the self-produced touch feels?

> **Check your prediction.** It feels more ticklish. The tickliness rating increased significantly with increasing delay and with increasing trajectory perturbation. Attenuation was weaker the less the touch corresponded to the movement in time and space. The authors proposed that how strongly a self-produced touch is attenuated depends on the error between the feedback predicted by an internal **forward model**, a computation that turns a motor command into a predicted sensory consequence, and the feedback that arrives.

The 2000 NeuroReport paper by the same group is a review, not a second experiment, that sets these results in that framework. Imaging cannot show that a region is needed, so the cerebellar account is suggestive. A forward model is one candidate explanation for the weak self-tickle, not a settled one.

## Mouse cortex: a movement signal and a mismatch signal

A response that changes during movement is not by itself a prediction. In awake mice, running more than doubles the visually evoked firing of most neurons in primary visual cortex (Niell and Stryker, 2010). A copy of the motor command, reafference from the movement or a correlated change of state could each produce it, so it is not yet an efference-copy pathway.

Two mouse results go further. Neurons in secondary motor cortex that project to auditory cortex drive suppression of auditory cortical responses during movement (Schneider, Nelson and Mooney, Nature, 2014). And neurons in primary visual cortex respond to mismatch between running and the visual flow it should produce (Keller, Bonhoeffer and Hübener, Neuron, 2012). The first is a movement signal reaching sensory cortex. The second is consistent with a failed prediction: the input did not match what the movement should have caused. It does not by itself identify the circuit that makes the prediction. For the visual pathway see [visual cortex](https://stemhop.com/lessons/visual-cortex/).

### Design problem

Pacinian afferents in freely moving mice fire during the animal’s own movements. Design the experiments that would tell you whether the central representation of that self-generated vibration is attenuated, and say what each design cannot show.

> **One way to do it.** First, record the same identified afferent class and a downstream station, in the same animal, during self-generated vibration and during externally applied vibration matched in amplitude and frequency content. Attenuation would appear as a smaller downstream response to the self-generated vibration for the same afferent drive. This cannot show how the attenuation arises, and it cannot rule out that the two stimuli differ in some way the matching missed.
>
> Second, replay the recorded self-generated stimulus passively and compare it with the active movement that produced it. A difference points to the motor command, movement-related feedback or a change of state that goes with moving, rather than the stimulus, but not to which.
>
> Third, impose a mismatch between the movement and its sensory consequence, as the vestibular work did, for example by adding an unexpected vibration during the movement, and ask whether attenuation disappears. If it does, the cancellation is conditional on a match. This supports a prediction mechanism in the tested circuit and condition, but a mismatch could also change attention or arousal, so a control for state is needed. None of the designs says whether the human tickle result uses the same mechanism.

## Going deeper

### When two cues may not share a cause

A sound and a flash far apart may have two causes, and then averaging them would be a mistake. Körding and colleagues (PLoS One, 2007) built an ideal-observer model that infers whether two cues come from the same location and also estimates the location. It predicted how people combined auditory and visual cues in two localization tasks, including the nonlinear way they did so.

### Why “independent” matters

With independent errors a second cue always lowers the variance. With correlated errors the best weights depend on the **covariance**, c, the shared part of the two errors: wV = (σH² − c)/(σV² + σH² − 2c). In the worked example (vision standard deviation (SD) 2 mm, touch SD 4 mm), suppose the two errors had a correlation of 0.5. Then c = 0.5 × 2 × 4 = 4 mm², wV = (16 − 4)/(4 + 16 − 2 × 4) = 1.0 and wH = 0.0, and the combined variance is 4 mm², no better than vision alone. Touch then adds nothing. A negative correlation works the other way: at −0.5 the combined variance falls to 1.71 mm², below the 3.2 mm² of the independent case.

### Collicular details

When neurons or trials are sorted by how weak their single-cue responses were, regression toward the mean can inflate the percentage for the weak group. In the 1986 sample, most multisensory cells had demonstrable descending projections.

### Electric fish and the model

The fish were 7 to 12 cm long, and they were neuromuscularly paralysed. The command occurred spontaneously at 2 to 5 Hz. The model had 20,000 granule-cell inputs.

### The Pacinian recordings in detail

Turecek and Ginty recorded in mice older than five and younger than ten weeks, of both sexes. They implanted glass electrodes at the fourth lumbar dorsal root ganglion, the cluster of sensory cell bodies beside the spinal cord.

### Movement, state and cortex

Niell and Stryker (Neuron, 2010) recorded from awake, head-fixed female mice, two to four months old, running or standing on a treadmill. Stimulus selectivity in primary visual cortex was unchanged during running. In that study the geniculate input did not show a locomotion-related increase in response magnitude, and some narrow-spiking cells were active during running yet suppressed by visual stimulation. Later recordings did find locomotion effects earlier in the pathway: Erisken and colleagues (Current Biology, 2014) reported that locomotion slightly but consistently increased firing rates in the dorsolateral geniculate nucleus.

Stringer and colleagues (Science, 2019) imaged on average about 11,000 excitatory neurons per session in awake head-fixed mice and found a high-dimensional ongoing signal partly related to behaviour, with visual stimuli adding representations in dimensions orthogonal to it. In separate brain-wide electrode recordings, reduced-rank regression, a method that predicts population activity from a small number of behavioural dimensions, found that predictions from video of the face saturated near 16 dimensions and explained an average of 21.9% of the total variance across areas. The caution in the main text applies: video alone cannot separate a motor-command copy, reafference and a correlated change of state, so none of these results is yet an efference-copy pathway.

### What the colliculus does with its output: looming

Yilmaz and Meister (Current Biology, 2013) showed freely moving mice, 8 to 12 weeks old, an overhead dark disk whose angular diameter grew from 2 to 20 degrees in 250 ms. The mice escaped or froze; in the initial flight assay three of ten began escaping within 250 ms, a subset and not an average. The circuit came later. Shang and colleagues (Science, 2015) identified parvalbumin-positive excitatory projection neurons in the mouse superior colliculus as a looming-related defensive pathway, and activating the projection to the parabigeminal nucleus triggered defensive responses. Here parvalbumin marks an excitatory projection neuron, not an interneuron.

In 2018 Shang and colleagues, working in adult male mice, separated two largely distinct projections of those neurons: to the parabigeminal nucleus, where activation produced escape followed by freezing in most mice tested, and to the lateral posterior thalamic nucleus, where activation produced immediate freezing. In the initial cohort, 44 of 58 mice showed escape followed by freezing. Bilaterally silencing either target shifted the pattern toward the one tied to the other. The colliculus is a system with competing outputs, not one fear switch, and these are laboratory behaviours of mice, not a measure of fear in people. For the retinal inputs see [retinal cell types](https://stemhop.com/lessons/retinal-cell-types/) and [direction selectivity](https://stemhop.com/lessons/direction-selectivity/); for restoring lost senses, [sensory prostheses](https://stemhop.com/lessons/sensory-prostheses/).

---

From STEMhop (https://stemhop.com), free lessons by Christopher Deppmann. Lesson text © the authors. Practice questions: https://stemhop.com/lessons/multisensory-and-prediction/
