Adaptive Stress Testing of Collision Avoidance Systems for Small UASs with Deep Reinforcement Learning

The next-generation Airborne Collision Avoidance System for smaller UASs (ACAS sXu) is
currently being developed and tested by the Federal Aviation Administration (FAA) to provide
detect-and-avoid capability for small unmanned aircraft operating beyond line-of-sight. Due
to the complexity and safety-critical nature of the system, safety validation is important not
only for the certification of the final system, but also for informing changes during the iterative
development process. In this paper, we analyze a prototype of ACAS sXu in simulated aircraft
encounters to discover scenarios of small near mid-air collisions (sNMACs), an important
safety event in which two aircraft come closer than 50 feet horizontally and 15 feet vertically.
Due to the size and complexity of the system as well as rarity of sNMAC events, traditional
methods such as Monte Carlo testing often require informed setup and targeting to elicit failures.
However, such a dependence on domain knowledge can be incompatible with the independent
verification and validation (IV&V) process, the aim of which is to discover unforeseen issues.
To address these challenges, we apply an accelerated validation method called adaptive stress
testing (AST) to find the most likely sNMAC scenarios without reliance on system introspection.
AST uses reinforcement learning to adapt the search towards the most promising areas of the
search space as it progresses. We use a state-of-the-art deep reinforcement learning algorithm,
proximate policy optimization, to more efficiently search the large and continuous state space.
We find that this approach significantly improves the performance of AST compared to a
prior approach based on Monte Carlo tree search. We perform experiments using AST to
find sNMAC events under various encounter configurations, varying parameters pertaining to
dynamics and coordination. Our experiments show AST to be very effective at finding sNMAC
scenarios. We summarize our findings, presenting high-level categories of discovered sNMACs
and specific examples of encounters in each category.