Severely Paralysed Woman Speaks Through Avatar Using Brain Signals
A groundbreaking development in brain-computer interface (BCI) technology has allowed a severely paralysed woman to communicate through an avatar, using her brain signals to generate speech and facial expressions. This breakthrough offers hope to individuals who have lost the ability to speak due to conditions such as strokes and amyotrophic lateral sclerosis (ALS). Until now, patients have had to rely on slow and frustrating speech synthesisers that require eye tracking or small facial movements, making natural conversation nearly impossible.
The latest technology involves the use of tiny electrodes implanted on the surface of the brain to detect electrical activity in the region responsible for speech and facial movements. These signals are then translated into speech and facial expressions by a digital avatar, allowing for more natural and fluid communication. Professor Edward Chang, leading the research at the University of California, San Francisco (UCSF), stated, “Our goal is to restore a full, embodied way of communicating, which is really the most natural way for us to talk with others. These advancements bring us much closer to making this a real solution for patients.”
The patient involved in the study is a 47-year-old woman named Ann, who has been severely paralysed since suffering a brainstem stroke over 18 years ago. Ann is unable to speak or type and currently relies on movement-tracking technology to select letters at a slow pace of up to 14 words per minute. She expressed hope that the avatar technology could enable her to work as a counsellor in the future.
To enable the avatar communication, the research team implanted a paper-thin rectangle containing 253 electrodes onto the surface of Ann’s brain, specifically targeting the region critical for speech. These electrodes intercepted the brain signals that would have controlled the muscles in her tongue, jaw, larynx, and face, had it not been for the stroke. Ann then worked closely with the team to train the system’s AI algorithm to detect her unique brain signals for different speech sounds by repeatedly repeating various phrases.
The computer successfully learned 39 distinctive sounds, and a Chat GPT-style language model was employed to translate the signals into intelligible sentences. The avatar’s voice was personalised to sound like Ann’s voice before her injury, based on a recording of her speaking at her wedding. However, the technology is not perfect, with a 28% error rate in word decoding during a test run involving over 500 phrases. The brain-to-text rate currently stands at 78 words per minute, compared to the average of 110-150 words spoken in natural conversation.
Despite these limitations, scientists believe that the recent advancements in accuracy, speed, and sophistication indicate that the technology is now practically useful for patients. Professor Nick Ramsey, a neuroscientist at the University of Utrecht in the Netherlands, who was not involved in the research, stated, “This is quite a jump from previous results. We’re at a tipping point.”
The next crucial step is to develop a wireless version of the BCI that can be implanted beneath the skull. Dr David Moses, co-author of the research and an assistant professor in neurological surgery at UCSF, explained, “Giving people the ability to freely control their own computers and phones with this technology would have profound effects on their independence and social interactions.”
The findings of this groundbreaking research have been published in the scientific journal Nature.
