Amyotrophic Lateral Sclerosis (ALS) patients experience limited mobility and communication, leading to less personal freedom and a lower quality of life. Stony Brook Department of Computer Science professors Fusheng Wang and Xiaojun Bi have been awarded a two-year, $200,000 grant by the ALS Association to advance their “EyeCanDo” technology, which is a multifunctional eye-gaze-based app to help ALS patients gain back some of that independence.
Originally developed for the 2018 Mount Sinai Medicine Hackathon, by students under their direction, EyeCanDo is an iOS app for iPhones and iPads with Apple’s TrueDepth camera system, which creates a detailed 3D map of a user’s face. This technology allows for the development of this novel ALS-focused eye-gaze-based assistive software product, which will enable patients to use their eye movements and visual attention to control smart home devices (such as lights or thermostats), listen to music, type, send messages and more.
While ALS patients typically develop progressive weakness that spreads gradually throughout all muscle groups, eye movement is usually spared. This, and the widespread use of iPhones and iPads, provides a unique opportunity for providing eye-gaze-based technologies to assist the patients without relying on specialized hardware.
Wang and Bi state that EyeCanDo will support a broad range of patient and caregiver needs, including both physical and psychosocial needs. They also believe that their app will benefit not only ALS patients but also other patients who lack mobility and ability to communicate due to disability secondary to other diseases such as multiple sclerosis, cerebral palsy, spinal cord injury, brain injuries and stroke, muscular dystrophy and others.
The development of EyeCanDo will include research into human-computer interaction and AI technologies to help achieve high accuracy and stability, multiple levels of communication and allow for a customized user experience by adapting the software to each individual patient. In addition to Wang and Bi, the EyeCanDo team includes other computer scientists with strong human-computer interaction, machine learning and biomedical informatics experience. Clinicians at the ALS Clinic at Stony Brook University Hospital and their patients will also be involved with evaluating the application.
About the Researchers
Fusheng Wang is an Associate Professor in the Department of Biomedical Informatics and the Department of Computer Science. He is also an affiliate faculty member of the AI Institute. His interests include: scalable data management systems, spatial and temporal data management and analytics, medical imaging informatics, healthcare and public health data analytics. The intersection of data management and biomedical informatics comprises much of his research. He seeks to develop novel methods and software systems to optimize the extraction, management, and mining of biomedical data with much improved efficiency, accuracy and interoperability to support biomedical research and the healthcare enterprise.
Xiaojun Bi is a Department of Computer Science Assistant Professor in the College of Engineering and Applied Sciences. His research interests include human-computer interaction, with a primary focus on building interactive systems, designing interaction techniques, and studying fundamental issues of user interface design on mobile devices. He has contributed a number of innovations for improving text entry in mobile computing, including bimanual gesture typing, optimizing the keyboard decoding algorithm for both correction and completion abilities, optimizing keyboard layouts, personalizing language models, and building automatic keyboard evaluation systems.