In an era marked by concerns over privacy invasion through modern technology, researchers in Department of Computer Science at Stony Brook University (SBU) have pioneered a groundbreaking solution to address eavesdropping. With the widespread presence of voice-activated devices fueling concerns about surreptitious surveillance, a transformative approach to wearable devices was designed to safeguard personal information while enhancing accessibility through silent communication.
Led by Professor Shubham Jain and her team in the PiCASSo Lab, their cutting-edge creations, MuteIt and Jawthenticate, herald a new frontier in privacy-conscious technology. These technologies not only redefine how users interact with their devices but also promise privacy in daily communications.
In an innovative departure from traditional voice-based interfaces, Jain, alongside her PhD students Tanmay Srivastava and Prerna Khanna; and collaborators Shijia Pan (UC Merced) and Phuc Nguyen (UMass Amherst), has spearheaded the development of ear-worn technology for unvoiced communications. MuteIt interprets silent speech through precise tracking of jaw movements. By allowing users to communicate with their devices without vocalizing, MuteIt represents a significant leap forward in accessibility technology. What sets MuteIt apart is its exceptional performance in noisy environments, surpassing traditional voice assistants. With MuteIt, users can interact discreetly with their devices, free from privacy concerns. In a study where participants had diverse speech accents, MuteIt recognized words with higher than 95% accuracy.
Further, Jain and her team developed Jawthenticate, utilizing the same ear-worn device, to demonstrate the distinctiveness of users' jaw movements as a biometric identifier. This research is the first to propose and validate the uniqueness of jaw movements. Jawthenticate captures the individualized combination of jaw movements and facial muscle vibrations for each user, allowing for the learning of speech-like features without the need for a microphone. These features are leveraged to authenticate the user's identity reliably. Remarkably, Jawthenticate can recognize users regardless of the language they speak or the content of their speech.
The innovative systems developed by Jain and her team ensure the preservation of privacy in a multitude of everyday scenarios. Whether in hospital environments, where hands-free and discreet communication is essential, or in routine daily activities, these solutions offer invaluable benefits. For instance, individuals concerned about disclosing sensitive information such as the last four digits of their SSN or DOB over the phone can rely on MuteIt to convey such details without fear of eavesdropping. With its lightweight design of under 20g and compatibility with devices like hearing aids and earphones, MuteIt holds tremendous potential for widespread adoption in various aspects of daily life.
Advancing their investigation into ear-worn systems, Jain and her team persist in refining these technologies to cater more effectively to individuals with specific needs. Jain emphasizes the transformative potential of these advancements, stating, "This system holds promise for tangible real-world impact. Many individuals face challenges in vocalizing due to diverse medical conditions, spanning from throat surgeries to ALS. Our innovations could potentially offer them a means to interact with their environment."
Read more about MuteIt and Jawthenticate in their ACM published articles.