Cornell's New Sonar Glasses to Deliver Silent-Speech Recognition Using Body Movements with 95% Accuracy

Isaiah Richard, Tech Times 07 April 2023, 09:04 pm

An innovation over at Cornell brings a plus to silent communication, with the new Sonar Glasses it developed using the EchoSpeech interface that could help in looking into the user's bodily movements to identify words. This mostly centers on the face and mouth recognition technology available in the glasses, with as much as 95% accuracy to what a user says.

It can perceive words that a user silently says, ideal for different situations and setups where sound is deadened or is unrecognizable due to noise pollution or other hindrances like speech disabilities.

Cornell's Sonar Glasses: Silent-Speech Recognition with 95% Accuracy

(Photo : Cornell)

Researchers at Cornell University have developed a pair of sonar glasses that can recognize silent speech by tracking the subtle movements of a speaker's mouth and face. The technology, which boasts an accuracy rate of 95%, could have significant implications for individuals with speech impairments or those working in noisy environments where traditional speech recognition systems struggle to function effectively.

The glasses, developed by Ph.D. student Ruidong Zhang from Cornell, use high-frequency sound waves to detect the movement of a speaker's mouth and convert them into words.

Zhang's published study debuts the latest wearable which also fits microphones and speakers "smaller than pencil erasers," paired with the AI-powered sonar to help detect words.

Communicating in Silence is Possible with the EchoSpeech

Previously, the glasses were equipped with a small camera that captures the speaker's facial expressions and lip movements, but the team has moved away from it. This is called the EchoSpeech interface, developed under the Smart Computer Interfaces for Future Interactions (SciFi) Lab.

It now uses acoustics sensing which is then analyzed using machine learning algorithms to capture what a user is trying to say. The system is trained on a large dataset of speech and facial expressions, which allows it to accurately predict what a speaker is saying after only several minutes of user training and gathering its needed data.

Speech Disabilities and their Solutions in the Present

Speech recognition is one technology that researchers in the world are also trying to bring to users, to answer problems and disabilities brought on by genetics or accidents to a person. At one time, Facebook (now Meta) worked on a brain-computer technology that could decode speech and hearing from brain activity.

There are also developments towards turning brain signals into speech, with thoughts that transform into words as per studies and research of the past.

The sonar glasses developed by Cornell researchers represent an exciting new development in the field of speech recognition technology.

By harnessing the power of passive sonar and machine learning algorithms, the glasses can accurately recognize silent speech with an impressive level of accuracy. With further refinement, this technology has the potential to transform the lives of individuals with speech impairments and provide a new way to communicate in noisy environments.