Perception contains a compelling mystery at its core: how can a seemingly simple bundle of nerve cells lead to so many diverse sensations, from the smell of freshly roasted coffee to the sight of an Impressionist painting? The seemingly automatic act of perceptual experience demands a surprisingly complicated input process, which takes place in two steps: first, there is transduction, in which a physical stimulus gets transformed into an electrical potential in the nervous system. Second (often concurrently), there is coding, in which that electrical signal gets reshaped into a format readable by downstream neurons.

An understanding of transduction and coding requires an understanding of how neurons communicate. A single neuron can be thought of as a simple computational system, which takes in electrochemical inputs via its dendrites, sums the inhibitory (negative) and excitatory (positive) inputs, and produces an electrochemical output. Typically, the neuron starts at a resting potential of about -70 mV, meaning the cell body is negatively charged with respect to the outside. As excitatory and inhibitory signals arrive, the electrical potential of the neuron changes. At the same moment, neurons nearby can detect this change in voltage. This is the first kind of electrical signal in the nervous system: a graded potential. Graded potentials are instantaneous and offer a continuous output range, but only work over short ranges, because a faraway charge is indistinguishable from a weak one.

Then, if enough excitatory signals are received, the neuron depolarizes, raising its voltage temporarily. This depolarization triggers the second kind of neural signal: an action potential (spike), which is an all-or-nothing signal. Action potentials can work over long ranges, but they take time to recover after sending a signal and can only express two levels of activity.

The brain as a whole can also be thought of as a computational system, which comes with neurons for input (sensory neurons), information processing (interneurons), and output (motor neurons). Perception starts with the system’s input layer: the sensory neurons, which take input from the environment and send output to interneurons in the cerebral cortex. Sensory neurons are the neurons responsible for transduction.

Transduction involves converting energy patterns in the environment into electrical signals in the nervous system. Each sensory system has its form of transduction: photoreceptors in the retina use stacks of photopigments to detect light. Hair cells in the ear use distortion to detect vibrations. Cutaneous receptors do the same to detect pressure on the skin. In each case, the receptor cells are converting another form of energy (electromagnetic, mechanical, etc.) into electrical energy – that is, an electric potential – representing the information that downstream interneurons will identify as sights, sounds, and other sensations.

As a case study, consider the photopigments inside of visual receptors in the retina. These are small molecules consisting of two parts – the opsin, which is a big anchor-like molecule, and the retinal, which is a small light-sensitive attachment. When the photopigment is exposed to light, the retinal changes shape, triggering a chain reaction that amplifies the cell’s change in electric potential. The electric potentials from the photoreceptors in turn excite bipolar cells in the retina, which in turn excite ganglion cells that – factoring in inhibitory inputs from horizontal cells – send action potentials along the optic nerve into the brain. While most other receptor neurons work in a similar way – using specialized molecular equipment to convert physical energy into electric potential – visual receptors are relatively unique in that they use graded potentials instead of action potentials. This might be because visual receptors in the retina do not require long-range communication, making the quick response times of graded potentials more attractive.

In general, sensory receptors need to use an electric potential to convey two kinds of information: the quality (e.g. smell, temperature, color) and the value (i.e. intensity) of each sensation. But how can a temporary change in the voltage of a cell – often just a discrete set of spikes – represent such complex information?

The quality of a sensation depends primarily on which part of the cortex get activated — regardless of the source of that activation. This is known as Müller’s Law of Specific Nerve Energies. For example, excitement in the visual cortex gets perceived as light, regardless of whether that stimulation arises from light, from dreaming, or from pressing on your eyes with your hands. Müller’s Law has some interesting implications: for one, if we could reroute input from the auditory neurons to the visual cortex, then sound would be perceived as light. Speculatively, the law might imply future therapies that repair the use of a sense by mapping an artificial device’s input to that part of the cortex – for example, by connecting a camera to the visual cortex of a blind person.

While coding for quality involves a one-to-one mapping from cortex area to sensory quality, coding the value of a sensation involves mapping a continuous range of input into a usable output for downstream neurons. Because graded potentials have limited usefulness at range, the continuous input range must often be represented in a discrete way by all-or-nothing action potentials. Some of the main ways to achieve this mapping are frequency coding, place coding, and ensemble coding — each method has its benefits and drawbacks.

With frequency coding, the firing rate of the neuron is proportional to the intensity of the stimulus. More specifically, many sensory systems use logarithmic coding to express a larger range of intensities, in which firing rate is proportional to the logarithm of the stimulus value. For example, each 10x increase in brightness might result in an increase of 20 spikes per second in a photoreceptor. This approach is useful because it only involves a single neuron, making it computationally cheap. However, because neurons need time to recover after each action potential, their firing rate is limited – and these slow firing rates lead to slow decoding times. Consider if the receiving neuron needed to distinguish between a 90 Hz input and a 100 Hz input. Over the course of 10 milliseconds, it would have to distinguish between 0.9 spikes and 1 spike (on average), which is quite difficult. Over the course of 1000 milliseconds, it could more easily distinguish between the 90 spikes and 100 spikes it receives from the upstream neuron. So frequency coding may be cheap — but it’s also slow.

Place encoding and ensemble encoding are both ways to encode and decode a value in a faster way by using more neurons. Place encoding uses each sensory neuron to encode a part of the range. When a specific neuron fires at a rate above its spontaneous rate, that indicates detection for its corresponding value. Unsurprisingly, this approach is expensive, requiring many neurons to measure a value at high resolution. A more efficient approach is ensemble coding, which gives each neuron an overlapping range of response values and infers the original value by taking a ratio between the level of activation for different neurons. While this strategy still requires many neurons, it achieves high resolutions without needing as many neurons as place encoding does.

Two other ways that the nervous system encodes values are recruitment and temporal encoding. In a recruitment scheme, more neurons firing represents a bigger value — this is what happens when more motor neurons firing in a muscle leads to a stronger activation of that muscle. In a temporal encoding scheme, receiving neurons monitor not just the rate of incoming spikes but also their pattern in time — are they clusters of quick spikes, or interspersed slower spikes? Although a temporal encoding scheme, like frequency coding, takes more time to decode, it’s likely the approach that our brains use to represent perceived pitch.

In practice, sensory systems often combine these approaches. For example, proprioception (the sense of body position) uses frequency coding despite its slowness because it doesn’t require many neurons. Meanwhile, the visual pathway tends to involve more expensive representations like ensemble coding, which allow us to respond to visual stimuli quickly and at high resolution.

An understanding of transduction and neural coding helps explain the first step of sensory perception: how information in the environment turn from patterns of energy — mechanical, chemical, thermal, electromagnetic — into electrical signals for the nervous system, and how these electrical signals represent quality and value.

Still, many questions remain: how do these signals get processed to form higher-level concepts, like a face or a musical chord? To what extent can these perceived stimuli be shaped by learning? And how does the electrochemical fact of perception give rise to the subjective experience of the same?