Want to schedule an appointment? Just ask your phone. Need to turn on your bedroom lights? Google Home has you covered.
Now a $49 billion market, voice-activated systems have gained popularity among consumers, thanks to their ability to automate and streamline mundane tasks. But for people with impaired speech, technologies that rely on voice commands have proved to be far from perfect.
That’s the impetus for Google’s newly formed Project Euphonia, part of the company’s AI for Social Good program. The project team is exploring ways to improve speech recognition for people who are deaf or have neurological conditions such as ALS, stroke, Parkinson’s, multiple sclerosis or traumatic brain injury.
Google has partnered with nonprofit organizations ALS Therapy Development Institute and ALS Residence Initiative (ALSRI) to collect recorded voice samples from people who have the neurodegenerative disease, one that often leads to severe speech and mobility difficulties.
For those with neurological conditions, voice-activated systems can play a key role in completing everyday tasks and conversing with loved ones, caregivers or colleagues. “You can turn on your lights, your music or communicate with someone. But this only works if the technology can actually recognize your voice and transcribe it,” says Julie Cattiau, a product manager at Google AI.
The company’s speech recognition technology utilizes machine learning algorithms that require extensive data training. “We have hundreds of thousands, or even millions, of sentences that people have read—and we use them as examples for the algorithms to learn how to recognize each,” says Cattiau. “But it’s not enough for people with disabilities.”
With Project Euphonia, the team will instead use voice samples from people who have impaired speech in the hope that the underlying system will be trained to understand inarticulate commands.
While the goal is to create technology that is more accessible for people with speech impediments, the end result is still unclear.
“It’s possible that we will have models that work for multiple people with ALS and other medical conditions,” says Cattiau. “It’s also possible that people, even just within ALS, sound too different to have such a machine learning model in place. And in that case, we may need to have a level of personalization so that each person has their own model.”
Google’s speech recognition technology can comprehend virtually any voice command for people without speech impairments, due to the large data set that has been available for training. But some uncertainty exists about how broadly speech technology will be able to understand and act on directives from those who have difficulty speaking. The Project Euphonia team has only a limited number of voice samples from people with speech impediments, which allows it to focus only on specific-use words and phrases such as “read me a book” or “turn off the lights.”
Though Cattiau’s team has collected tens of thousands of recorded phrases, she says it needs hundreds of thousands more. That’s partly why Google CEO Sundar Pichai unveiled this project at the company’s annual developer conference in May.
“We are working hard to provide these voice-recognition models to the Google Assistant in the future,” he said, calling on people with slurred and impaired speech to submit their voice samples.
“Impaired speech is a very difficult data set to put together. It’s not as simple as asking people to record phrases, and there’s no data set just lying around,” Cattiau says. “We have to first put it together, and that’s a lot of work.”
Perhaps the most groundbreaking of Project Euphonia’s initiatives is its work on new interactive AI systems for people who are completely nonverbal. Also in its early stages, these systems are being trained to detect gestures, vocalizations and facial expressions, which can then trigger certain actions like sending or reading a text message.
“We want to cover the full spectrum of people—and not only those who can still speak,” says Cattiau. Although Project Euphonia is still in its infancy, it could eventually have a great impact on those with disabilities, giving them the freedom and flexibility to live independently.Follow me on Twitter.
-Ruth Umoh; Forbes Staff