New Software Testing Methods Improve Aircraft Safety and Help Save Lives

Computer Science Professor receives award from the Air Force's Young Investigator Research Program

Automation is increasing in our daily lives, as well as the lives of those serving in the U.S. Air Force. Defense contractors are using the latest research in their unmanned aerial systems such as artificial intelligence (AI) adapting to battlefield conditions. For example, the Air Force intends to create a family of unmanned aerial drones that utilizes AI technology to function as a “wingman” to human-piloted fighter jets, such as F-15s. Such drones are complex cyber-physical systems. Such systems include software, but also have physical world components where sensors can sense physical signals and model complex interactions such as aerodynamics.

Professor Stanley Bak of the Department of Computer Science has recently received an award from the Air Force Office of Scientific Research (AFOSR) in their prestigious Young Investigator Program (YIP). He will receive a grant of $450,000 over three years to study the ‘Science of Fuzz Testing Autonomous Cyber-Physical Systems.’ Professor Bak’s YIP project applies to testing of complex autonomous cyber-physical systems such as unmanned drones. Professor Bak will pioneer this work via ‘Fuzz Testing,’ a technique that has been used for finding bugs or vulnerabilities in software by using directed randomness. This approach allows significantly more tests to be run compared with a manual approach and has been effectively used by large software companies.
 

While fuzz testing has been applied primarily to software security thus far, the goal of Bak’s YIP research project is to extend this powerful technique into other domains, in particular to cyber-physical systems like military drones. Such drones have dozens of autonomous functions that could interact with each other. While individually testing these components is important, it is much more challenging to manually design tests to explore the interactions among components. The testing problem becomes even more difficult when sensors can fail or provide degraded information (when the
sun reflects into a camera), or if other rare events occur during the handoff of control between modules. These systems could also include machine learning components such as neural networks, which could benefit from tailored strategies to improve testing efficiency.

“Many systems used by the Air Force are becoming increasingly autonomous, as complex functions traditionally performed by pilots are now being done by software,” said Professor Bak. “Behaviors like waypoint following or ground-collision avoidance may interact in unanticipated ways, and testing is essential to detect problems ahead of time.”

Professor Bak’s F-16 verification benchmark library featuring a GIF which simulates an aircraft performing autonomous maneuvers

Samir Das, Chair of the Department of Computer Science, said, “Professor Bak is a renowned cyber-physical systems expert who significantly contributed to the U.S. Air Force’s capabilities in the past. We are proud of his recognition in the YIP program.”

“Protecting Air Force service members in hazardous situations is an incredibly important contribution we can make as civilians,” said Robert Kutka, Acting Dean, College of Engineering and Applied Sciences. “Through his successful proposal for the YIP award, Professor Bak and Stony Brook will play a part in supporting our troops. Given the increasing usage of AI and automation in daily life, the outcomes of his research will likely have mass adaptation beyond the Air Force.”

Professor Bak previously worked in the verification and validation group in the Aerospace Systems Directorate and in the Information Directorate of the U.S. Air Force Research Lab. Eventually, AI capabilities may be intelligent enough that they can autonomously compile information and complete tasks to aid in a reduction of fighter pilot workload. Such systems can learn from prior experiences on how to best adapt to current circumstances. Professor Bak’s testing research will be essential to furthering AI use in safety-critical situations and for protection of Air Force personnel and material assets against combat hazards.

Bio: Stanley Bak is an assistant professor in the Department of Computer Science, part of the College of Engineering and Applied Sciences, investigating the verification of autonomy, cyber-physical systems, and neural networks. He strives to develop practical formal methods that are both scalable and useful, which demands developing new theory, programming efficient tools and building experimental systems.

-Andrew Solar-Greco