Feature

MIKE KAI CHEN/THE NEW YORK TIMES/REDUX/EYEVINE

Brain–computer interface technology is helping people with paralysis to speak — and providing lessons about brain anatomy.

MIND-READING DEVICES ARE

REVEALING THE BRAIN’S SECRETS
Interfaces that decode neural activity can restore people’s abilities to move and
speak. They’re also changing our understanding of brain function. By Miryam Naddaf

M
oving a prosthetic arm. brain is organized, and with greater resolution working on a BCI for speech.
Controlling a speaking avatar. than most other methods. The devices also allow measurements over
Typing at speed. These are all Scientists have used these opportunities much longer time spans than classical tools do,
things that people with paral- to learn some basic lessons about the brain. says Edward Chang, a neurosurgeon at the Uni-
ysis have learnt to do using Results are overturning assumptions about versity of California, San Francisco. “BCIs are
brain–computer interfaces brain anatomy, for example, revealing that really pushing the limits, being able to record
(BCIs) — implanted devices that regions often have much fuzzier boundaries over not just days, weeks, but months, years at
are powered by thought alone. and job descriptions than was thought. Such a time,” he says. “So you can study things like
These devices capture neural activity using studies are also helping researchers to work learning, you can study things like plasticity,
dozens to hundreds of electrodes embedded out how BCIs themselves affect the brain and, you can learn tasks that require much, much
in the brain. A decoder system analyses the crucially, how to improve the devices. more time to understand.”
signals and translates them into commands. “BCIs in humans have given us a chance to
Although the main impetus behind the work record single-neuron activity for a lot of brain Recorded history
is to help restore functions to people with areas that nobody’s ever really been able to do The idea that the electrical activity of the
paralysis, the technology also gives research- in this way,” says Frank Willett, a neuroscien- human brain could be recorded first gained
ers a unique way to explore how the human tist at Stanford University in California who is support 100 years ago. German psychiatrist

706 | Nature | Vol 626 | 22 February 2024

Hans Berger attached electrodes to the revealed some surprising lessons about its find out whether that’s important” for making
scalp of a 17-year-old boy whose surgery for function along the way. movements, he says.
a brain tumour had left a hole in his skull. Movement relies on a lot of coordination,
When Berger recorded above this opening, Fuzzy boundaries and brain activity has to synchronize it all,
he made the first observation of brain oscilla- Textbooks often describe brain regions as explains Ramsey. Holding out an arm affects
tions and gave the measurement a name: the having discrete boundaries or compartments. balance, for instance, and the brain has to man-
EEG (electroencephalogram). But BCI recordings suggest that this is not age those shifts across the body, which could
Researchers immediately saw that record- always the case. explain the dispersed activity. “There’s a lot
ing from inside the brain could be even more Last year, Willett and his team were using of potential in that kind of research that we
valuable; Berger and others used surgery to a BCI implant for speech generation in a per- haven’t thought of before,” he says.
place electrodes on the surface of the cortex to son with motor neuron disease (amyotrophic To some scientists, these fuzzy anatomical
study the brain and diagnose epilepsy. Record- lateral sclerosis). They expected to find that boundaries are not surprising. Our under-
ing from implanted electrodes is still a stand- neurons in a motor control area called the pre- standing of the brain is based on average meas-
ard method for pinpointing where epileptic central gyrus would be grouped depending urements that paint a generalized image of
seizures begin, so that the condition can be on which facial muscles they were tuned to — how this complex organ is arranged, says Luca
treated using surgery. jaw, larynx, lips or tongue. Instead, neurons Tonin, an information engineer at the Univer-
Then, in the 1970s, researchers began to use with different targets were jumbled up4. “The sity of Padua in Italy. Individuals are bound to
signals recorded from further inside animal anatomy was very intermixed,” says Willett. diverge from the average.
brains to control external machines, giving They also found that Broca’s area, a brain “Our brains look different in the details,”
rise to the first implanted brain–machine region thought to have a role in speech pro- says Juan Álvaro Gallego, a neuroscientist at
interfaces. duction and articulation, contained little to Imperial College London.
In 2004, Matt Nagle, who was paralysed no information about words, facial move- To others, findings from such a small num-
after a spinal injury, became the first person ments or units of sound called phonemes. “It ber of people should be interpreted with cau-
to receive a long-term invasive BCI system seems surprising that it’s not really involved tion. “We need to take everything that we’re
that used multiple electrodes to record in speech production per se,” says Willett. learning with a grain of salt and put it in con-
activity from individual neurons in his pri- text,” says Chang. “Just because we can record
mary motor cortex1. Nagle was able to use his “BCIs are really pushing the from single neurons doesn’t mean that’s the
system to open and close a prosthetic hand, most important data, or the whole truth.”
and to perform basic tasks with a robotic arm.
limits, being able to record
Researchers have also used EEG readings over not just days, weeks, but Flexible thinking
— collected using non-invasive electrodes months, years at a time.” BCI technology has also helped researchers to
placed on a person’s scalp — to provide signals reveal neural patterns of how the brain thinks
for BCIs. These have allowed paralysed peo- and imagines.
ple to control wheelchairs, robotic arms and Previous findings using other methods had Christian Herff, a computational neurosci-
gaming devices, but the signals are weaker hinted at this more nuanced picture (see, for entist at Maastricht University, the Nether-
and the data less reliable than with invasive example, ref. 5). lands, studies how the brain encodes imagined
devices. In a 2020 paper about motion6, Willett and speech. His team developed a BCI implant
So far, about 50 people have had a BCI his colleagues recorded signals in two people capable of generating speech in real time
implanted, and advances in artificial intel- with different levels of movement limitation, when participants either whisper or imagine
ligence, decoding tools and hardware have focusing on an area in the premotor cortex speaking without moving their lips or making a
propelled the field forwards. that is responsible for moving the hands. sound8. The brain signals picked up by the BCI
Electrode arrays, for instance, are becom- They discovered while using a BCI that the device in both whispered and imagined speech
ing more sophisticated. A technology called area contains neural codes for all four limbs were similar to those for spoken speech. They
Neuropixels has not yet been incorporated together, not just for the hands, as previously share areas and patterns of activity, but are not
into a BCI, but is in use for fundamental presumed. This challenges the classical idea the same, explains Herff.
research. The array of silicon electrodes, that body parts are represented in the brain’s That means, he says, that even if someone
each thinner than a human hair, has nearly cortex in a topographical map, a theory that can’t speak, they could still imagine speech
1,000 sensors and is capable of detecting elec- has been embedded in medical education for and work a BCI. “This drastically increases the
trical signals from a single neuron. Research- nearly 90 years. people who could use such a speech BCI on a
ers began using Neuropixels arrays in animals “That’s something that you would only see clinical basis,” says Herff.
seven years ago, and two papers published if you’re able to record single-neuron activity The fact that people with paralysis retain the
in the past three months demonstrate their from humans, which is so rare,” says Willett. programmes for speech or movement, even
use for questions that can be answered only Nick Ramsey, a cognitive neuroscientist when their bodies can no longer respond, helps
in humans: how the brain produces and at University Medical Center Utrecht in the researchers to draw conclusions about how
perceives vowel sounds in speech2,3. Netherlands, made similar observations plastic the brain is — that is, to what extent it
Commercial activity is also ramping up. In when his team implanted a BCI in a part of can reshape and remodel its neural pathways.
January, the California-based neurotechnol- the motor cortex that corresponds to hand It is known that injury, trauma and disease
ogy company Neuralink, founded by entre- movement7. The motor cortex in one hemi- in the brain can alter the strength of con-
preneur Elon Musk, implanted a BCI into a sphere of the brain typically controls move- nections between neurons and cause neural
person for the first time. As with other BCIs, ments on the opposite side of the body. But circuits to reconfigure or make new connec-
the implant can record from individual neu- when the person attempted to move her right tions. For instance, work in rats with spinal
rons, but unlike other devices, it has a wireless hand, electrodes implanted in the left hem- cord injuries has shown that brain regions
connection to a computer isphere picked up signals for both the right that once controlled now-paralysed limbs
And although the main driver is clinical hand and the left hand, a finding that was can begin to control parts of the body that
benefit, these windows into the brain have unexpected, says Ramsey. “We’re trying to are still functional9.

Nature | Vol 626 | 22 February 2024 | 707

Feature
One possible explanation could be that the
brain becomes more efficient at controlling
the device and requires fewer neural resources
to do the tasks, says Marchesotti.
Studying how the brain behaves during
BCI use is an emerging field, and researchers
hope it will both benefit the user and improve
BCI systems. For example, recording activity
across the brain allows scientists to detect
whether extra electrodes are needed in other
decoding sites to improve accuracy.
Understanding more about brain organ-
ization could help to build better decoders
and prevent them making errors. In a pre-
print posted last month13, Ramsey and his
colleagues showed that a speech decoder can
become confused between a user speaking a
sentence and listening to it. They implanted
BCIs in the ventral sensorimotor cortex — an

SILVIA MARCHESOTTI
area commonly targeted for speech decoding
— in five people undergoing epilepsy surgery.
They found that patterns of brain activity seen
when participants spoke a set of sentences
Scientists have studied how brain–computer interfaces, such as this non-invasive cap, closely resembled those observed when they
change brain activity. listened to a recording of the same sentences.
This implies that a speech decoder might not
But BCI studies have muddied this picture. technique known as functional electrical be able to differentiate between heard and
Jennifer Collinger, a neural engineer at the Uni- stimulation (FES). Whenever the BCI decoded spoken words when trying to generate speech.
versity of Pittsburgh in Pennsylvania, and her the participants’ attempts to extend their The scope of current BCI research is still lim-
colleagues used an intracortical BCI in a man hands, it stimulated the muscles that control ited, with trials recruiting a very small number
in his 30s who has a spinal cord injury. He can wrist and finger extension. Participants in of participants and focusing mainly on brain
still move his wrist and elbow, but his fingers the control group had the same set-up, but regions involved in motor function.
are paralysed. received random electrical stimulation. “You have at least tenfold as many research-
Collinger’s team noticed that the original Using EEG imaging, Millán’s team found ers working on BCIs as you have patients using
maps of the hand were preserved in his brain10. that the participants using BCI-guided FES BCIs,” says Herff.
When the man attempted to move his fingers, had increased connectivity between motor Researchers value the rare chances to record
the team saw activity in the motor area, even areas in the affected brain hemisphere com- directly from human neurons, but they are
though his hand did not actually move. pared with the control group. Over time, the driven by the need to restore function and
“We see the typical organization,” she says. BCI–FES participants became able to extend meet medical needs. “This is neurosurgery,”
“Whether they have changed at all before or their hands, and their motor recovery lasted says Collinger. “It’s not to be taken lightly.”
after injury, slightly, we can’t really say.” That for 6–12 months after the end of the BCI-based To Chang, the field naturally operates as a
doesn’t mean the brain isn’t plastic, Collinger rehabilitation therapy. blend of discovery and clinical application.
notes. But some brain areas might be more flex- “It’s hard for me to even imagine what our
ible in this regard than others. “For example, What BCIs do to the brain research would be like if we were just doing
plasticity seems to be more limited in sensory In Millán’s study, the BCI helped to drive learning basic discovery or only doing the BCI work
cortex compared to motor cortex,” she adds. in the brain. This feedback loop between human alone,” he says. “It seems that both really are
In conditions in which the brain is damaged, and machine is a key element of BCIs, which critical for moving the field forwards.”
such as stroke, BCIs can be used alongside can allow direct control of brain activity. Par-
other therapeutic interventions to help train ticipants can learn to adjust their mental focus Miryam Naddaf is a correspondent for Nature
a new brain area to take over from a damaged to improve the decoder’s output in real time. in London.
region. In such situations, “people are perform- Whereas most research focuses on optimiz-
1. Hochberg, L. R. et al. Nature 442, 164–171 (2006).
ing tasks by modulating areas of the brain that ing BCI devices and improving their coding 2. Leonard, M. K. et al. Nature https://doi.org/10.1038/
originally were not evolved to do so”, says José performance, “little attention has been paid to s41586-023-06839-2 (2023).
3. Khanna, A. R. et al. Nature https://doi.org/10.1038/
del R. Millán, a neural engineer at the University what actually happens in the brain when you use
s41586-023-06982-w (2024).
of Texas at Austin, who studies how to deploy the thing”, says Silvia Marchesotti, a neuroengi- 4. Willett, F. R. et al. Nature 620, 1031–1036 (2023).
BCI-induced plasticity in rehabilitation. neer at the University of Geneva, Switzerland. 5. Tate, M. C. et al. Brain 137, 2773–2782 (2014).
6. Willett, F. R. et al. Cell 181, 396–409 (2020).
In a clinical trial, Millán and his colleagues Marchesotti studies how the brain changes
7. Vansteensel, M. J. et al. N. Engl. J. Med. 375, 2060–2066
trained 14 participants with chronic stroke — when people use a BCI for language genera- (2016).
a long-term condition that begins 6 months tion — looking not just in the regions where the 8. Angrick, M. et al. Commun. Biol. 4, 1055 (2021).
9. Ghosh, A. et al. Nature Neurosci. 13, 97–104 (2010).
or more after a stroke, marked by a slowdown BCI sits, but more widely. Her team found that,
10. Ting, J. E. et al. J. Neurophysiol. 126, 2104–2118 (2021).
in the recovery process — to use non-invasive when 15 healthy participants were trained to 11. Biasiucci, A. et al. Nature Commun. 9, 2421 (2018).
BCIs for 6 weeks11. control a non-invasive BCI over 5 days, activ- 12. Bhadra, K., Giraud, A. L. & Marchesotti, S. Preprint at
bioRxiv https://doi.org/10.1101/2023.09.11.557181 (2023).
In one group, the BCI was connected to a ity across the brain increased in frequency
13. Schippers, A., Vansteensel, M. J., Freudenburg,
device that applied electric currents to acti- bands known to be important for language Z. V. & Ramsey, N. F. Preprint at medRxiv https://doi.
vate nerves in paralysed muscles, a therapeutic and became more focused over time12. org/10.1101/2024.01.21.23300437 (2024).

708 | Nature | Vol 626 | 22 February 2024