Liam Drew
The rise of brain-reading technology: what you need to know
As implanted devices and commercial headsets advance, what will the real-world impacts be?
A participant in Edward Chang’s study of speech neuroprostheses attempts to silently speak a sentence.
Ann, who was left paralysed by a stroke, uses a brain–computer interface to translate brain signals into the speech and movement of an avatar. Credit: Noah Berger
In a laboratory in San Francisco, California, a woman named Ann sits in front of a huge screen. On it is an avatar created to look like her. Thanks to a brain–computer interface (BCI), when Ann thinks of talking, the avatar speaks for her — and in her own voice, too.
In 2005, a brainstem stroke left Ann almost completely paralysed and unable to speak. Last year, neurosurgeon Edward Chang, at the University of California, San Francisco, placed a grid of more than 250 electrodes on the surface of Ann’s brain, on top of the regions that once controlled her body, face and larynx. As Ann imagined speaking certain words, researchers recorded her neural activity. Then, using machine learning, they established the activity patterns corresponding to each word and to the facial movements Ann would, if she could, use to vocalize them.
The system can convert speech to text at 78 words per minute: a huge improvement on previous BCI efforts and now approaching the 150 words per minute considered average for regular speech1. Compared with two years ago, Chang says, “it’s like night and day”.
In an added feat, the team programmed the avatar to speak aloud in Ann’s voice, basing the output on a recording of a speech she made at her wedding. “It was extremely emotional for Ann because it was the first time that she really felt that she was speaking for almost 20 years,” says Chang.
This work was one of several studies in 2023 that boosted excitement about implantable BCIs. Another study2 also translated neural activity into text at unprecedented speed. And in May, scientists reported that they had created a digital bridge between the brain and spinal cord of a man paralysed in a cycling accident3. A BCI decoded his intentions to move and directed a spinal implant to stimulate the nerves of his legs, allowing him to walk.
“There’s a lot of energy, and it’s super exciting,” Chang says. “I think that we’re going to cross a really important threshold in the next five years: coming out of proof of principles into new therapies.”
Mind-reading machines are coming — how can we keep them in check?
Companies in the field are also making strides: in September the neurotechnology company Neuralink, founded by entrepreneur Elon Musk, invited people living with paralysis to volunteer to be the first recipients of its implantable BCI.
The quest to commercialize BCIs, however, is still in its infancy. So far, systems are tailored to individuals, but commercialization will require robust, reliable and safe BCIs that can be scaled up. “You cannot have a PhD engineer in the home of every single patient with a BCI,” says Tom Oxley, chief executive of Synchron, a BCI company in Brooklyn, New York.
Alongside advances in implantable devices, a parallel commercial ecosystem of wearable brain-reading devices is growing. These measure users’ brain activity — at much lower resolution than implanted devices — to potentially enhance mental health, productivity or sleep, or to transform how people interact with computers.
Together, these advances are accelerating efforts to guide and regulate neurotechnology. This month, for instance, member states of UNESCO — the United Nations cultural and scientific organization — will vote on whether to develop international guidelines and policy recommendations for the use of this technology.
As progress generates headlines, there is no shortage of grand claims. Consumer-targeted bioinformatics company EMOTIV in San Francisco describes its team as “decoders of the human experience”. In 2020, Musk told podcaster Joe Rogan that Neuralink’s BCI “could, in principle, fix almost anything that’s wrong with the brain”.
“We need to have more conversation,” says Chang, “and try to reduce the hype and focus on the things that are actually really relevant.”
Decoding the brain
All brain-reading technologies, whether implants or headsets, operate on the same basic principles: they record neural activity — usually electrical activity — associated with a function such as speech or attention; interpret what that activity means; and use it to control an external device or simply provide it as information to the user.
Implanted BCIs record more information-rich brain signals than do external ones. But these experimental devices are intended only for use by people in whom potential clinical benefits outweigh the risks of, for example, brain injury or infection. Only around 50 people have received such implants long-term.
A staff member demonstrates a brain-computer interface at the World Robot Conference 2023 in Beijing.
An attendee at a conference in Beijing demonstrates a brain–computer interface headset.Credit: Wang Yuguo/Xinhua News Agency/eyevine
Most devices worn on the scalp use a common method called electroencephalography (EEG) to detect tiny electrical fields that pass through the skull, reflecting the average firing of many millions of neurons spread over substantial volumes of brain.
EEG is routinely used clinically to monitor epilepsy and sleep, and in the lab to study a range of brain functions. Commercial efforts centre on using EEG signals to monitor psychological states such as focus, calmness, agitation and drowsiness.
Consumer-targeted companies have yet to create a ‘killer app’ — an application so desirable that sales take off drastically. But for implantable devices, an alluring application is clear: helping people living with paralysis to restore communication or autonomy.
Improving implants
Various companies are developing and commercializing implanted BCIs. Closest to the clinic are five US companies: Neuralink; Synchron; Blackrock Neurotech in Salt Lake City, Utah; Paradromics in Austin, Texas; and Precision Neuroscience in Manhattan, New York. China is also heavily invested in this field and European companies are emerging.
Blackrock Neurotech, Paradromics and Neuralink have developed electrode systems that penetrate the brain’s cortex to record from individual neurons. Paradromics’ chief executive, Matt Angle, says academic research has shown that the more neurons that are recorded from, the more accurately and quickly thoughts can be decoded.
The interfaces from Blackrock Neurotech and Paradromics are grids of hundreds of stiff, straight electrodes, and multiple arrays can be implanted in a single person. Blackrock Neurotech’s array was first implanted long-term in a person 19 years ago4, and has been a mainstay of BCI research ever since. Paradromics’ array is undergoing testing in sheep.
Decoding the business of brain-computer interfaces
Neuralink’s implant — so far tested in monkeys — consists of multiple long, flexible polymer threads. These contain many recording sites and are implanted deeper in the cortex than are stiff electrode arrays.
Conversely, Synchron and Precision Neuroscience use electrodes that sit on the surface of the brain, such as those deployed in Chang’s study. “Our whole philosophy is around minimally invasive deployment of electrodes that don’t damage the brain,” says Ben Rapoport, co-founder of Precision Neuroscience. This includes being able to easily remove them, he says.
Synchron’s BCI contains just 16 electrodes and bucks the trend of chasing ever more bandwidth. Termed a stentrode, it is a hybrid of a blood-vessel stent and an electrode array. It is implanted without neurosurgery, by pushing the device up through the jugular vein until it sits in the blood vessel that lies beneath the brain’s motor cortex, the region that formulates a person’s intentions to move.
The stentrode’s low bandwidth cannot decode thoughts, but it enables users to control a smartphone — a potentially transformative gain of autonomy5. “You have to choose what you’re going to optimize for,” says Oxley.
Between them, these companies are a hive of early-stage clinical activity. This year, Neuralink was cleared to begin human trials of its device; Precision Neuroscience tested its electrodes in humans for the first time (recording for 15 minutes during operations to remove brain tumours). And all five companies have now gained breakthrough device status — an accelerated route to clinical approval — from the US Food and Drug Administration (FDA).
Synchron is closest to potentially gaining approval. This year, the tenth and final volunteer joined the company’s initial feasibility studies, in which people with severe paralysis are using Synchron’s system at home. In September, the company achieved the goal of having someone with a newly implanted stentrode device follow software instructions to set up the BCI without assistance from Synchron staff. The other four companies hope to proceed through feasibility trials in the coming years.
Tim Denison, an engineer at the University of Oxford, UK (who has consulted for Synchron), has worked in neurotechnology for 20 years, often focusing on brain stimulation as a treatment for neuropsychiatric conditions. Denison says advances in brain reading could make a huge difference in guiding therapeutic stimulation, if devices can identify signatures of disease — or signs of recovery.
But Denison’s long experience makes him cautious. In some situations, “I had very high hopes and the technology didn’t come through”, he says, “And that’s very humbling.”
He stresses that making devices reliable, practicable and affordable is just as crucial to their success as the scientific advances. Given the scarcity of neurosurgeons worldwide, Denison says, one of the most significant things Neuralink has done is to create a robot that surgically implants its device.
Consumer focus
Developers of non-invasive consumer brain-reading headsets face a different set of hurdles. The current commercial ecosystem consists of a few small, relatively established companies, dozens of start-ups and various research departments in big tech companies.
The brain-reading devices helping paralysed people to move, talk and touch
“The three big limitations of consumer neurotechnology have been the form that they’ve taken, the applications that they’ve offered and the quality of the signal that you could get out of them,” says Nita Farahany, a legal scholar specializing in this field at Duke University in Durham, North Carolina.
Despite some success under controlled lab conditions, EEG cannot decode users’ thoughts. And although some products — especially for gaming — use EEG to control external technology, it is currently rather a slow and effortful process.
EEG is better at giving a general indication of someone’s psychological state. In different states — such as sleep or focused working — neuronal firing tends to coalesce into oscillatory waves at distinct frequencies. Sleep, for example, is defined by slow delta waves; relaxation is associated with intermediate theta waves; and attention with faster alpha waves.
Many applications aim to make users consciously aware of their brain state — through some form of interface — in order to help them change it.
Several companies offer EEG-sensing products such as headbands, headsets or earphones that they say nudge users towards deeper meditative states, or help people to enter more focused and more productive states. In 2022, Liverpool Football Club announced that German neurotech company Neuro11, based in Potsdam, had helped the club’s players to learn to achieve calm, focused states in pressurized situations and had aided their performance — although researchers warn that there can be large placebo effects with such interventions.
Liverpool’s Trent Alexander-Arnold wears sensors on his head during a media day and training session.
Trent Alexander-Arnold, who plays for Liverpool Football Club, wears a headset that can detect electrical brain activity during a training session.Credit: Jon Super/AP Photo/Alamy
Some products aim to manipulate brain waves directly in hopes of changing a person’s mental state. Andrew Jackson, a neuroscientist at Newcastle University, UK, co-founded Neudio, a start-up company that uses an algorithm that records a user’s EEG and, in real time, generates synthetic music that aims to entrain brain activity to induce relaxation or focus. Other companies are using similar approaches to improve, for example, sleep quality.
But Farahany suspects that this technology will become mainstream for other uses. “I think the things that will really make neural interfaces ubiquitous is using them to replace existing peripheral devices — and in virtual reality and augmented reality.”
Companies such as Meta and Apple have already launched headsets that include, for example, eye-tracking technology — heralding a shift towards more-immersive computing experiences, says Farahany. In July, Apple was granted a patent to incorporate EEG sensors into its earbuds, called Airpods.
Significant questions remain about the quality of EEG signals that consumer devices can record — especially when a user is moving — and how this will limit applications. But these technologies could mean more than just new ways to enhance personal computing experiences. They raise questions of whether someone’s brain data — and even their mental privacy — will be commodified.
Sanctuary for sale
As brain-reading neurotechnologies accelerate, ethicists and regulators are increasingly asking what unique risks these devices pose. “The brain is not just another organ of the body; it is the organ that generates the human mind. This should be the sanctuary of our identity,” says Rafael Yuste, a neuroscientist at Columbia University in New York City. “You need to shield that, you cannot just go in and start banking and selling brain data.”
Abandoned: the human cost of neurotechnology failure
Implanted medical technology might create ethical issues. For example, given that artificial-intelligence (AI) software helps to turn users’ brain activity into decisions, there are questions regarding users’ agency and culpability. It is also unclear what would happen to people if the manufacturer of their implant ceases to operate. But the general view is that existing medical regulations can largely guide tech development and use. For consumer devices, however, current regulations leave worrying gaps, says Farahany.
In her book, The Battle for Your Brain, which was released in March, Farahany describes how, in China, schoolchildren’s attention has been monitored using EEG headsets made by US software company BrainCo, and how certain employers, across multiple countries, are monitoring their employees. The ethics vary with the situation: such tracking could be valuable for noticing when long-distance drivers become drowsy, but thornier if employers use the technology to police employees’ concentration levels.
Critics argue that some claims made about EEG’s ability to reveal individuals’ private thoughts are overblown — and that data gathered from people’s online behaviour are much more revealing. However, Yuste draws a hard line between overt behaviour and private mental activity. He says rapid improvements in AI decoding and non-invasive hardware “make the fight for your mental privacy much more urgent”.
Yuste and Farahany think existing human-rights treaties need updating to protect citizens against the misuse of neurotechnologies. Yuste advocates for a new class of rights termed neurorights — which, he says, would protect mental privacy; prevent personality-changing manipulations; and guard against biases in the algorithms behind neurotech.
Farahany argues for a wider right to ‘cognitive liberty’ — protection from both neurotechnology and a range of digital technologies that can manipulate people’s minds and behaviour.
Multiple organizations are exploring how neurotech should be regulated. Since 2019, UNESCO, the Organisation for Economic Co-operation and Development and the UK Regulatory Horizons Council have each issued recommendations or reports. This month’s vote at UNESCO will decide whether the agency should produce an extensive international framework for neurotech governance.
But ethicists ultimately want to see principles become enshrined in law. One solution is to modify international human-rights treaties; the UN’s human-rights council met in August to discuss neurotechnology.
Chile is currently the only nation that has legislation protecting neurorights. In 2021, it changed its constitution to guard against problematic uses of neurotechnology. This year, the senator who has championed neurorights the most, Guido Girardi, successfully sued EMOTIV in Chile’s Supreme Court for violating the country’s rules on collecting and using neural data after he imported and used one of its devices.
Elsewhere, the governments of Brazil, Mexico, Spain and Australia are discussing how they might create legislation for neurotechnology.
Farahany is buoyed by the fact that — unlike attempts to regulate social media and AI, which happened only once these technologies started being used on a massive scale — conversations about neurotechnology are happening before its tipping point. “Internationally, people seem to care about doing it right and doing it ethically.”
Nature 623, 241-243 (2023)
doi: https://doi.org/10.1038/d41586-023-03423-6
References
- Metzger, S. L. et al. Nature 620, 1037–1046 (2023).
- Willett, F. R. et al. Nature 620, 1031–1036 (2023).
- Lorach, H. et al. Nature 618, 126–133 (2023).
- Hochberg, L. R. et al. Nature 442, 164–171 (2006).
- Mitchell, P. et al. JAMA Neurol. 80, 270–278 (2023).
Leave a Reply