Machine Learning to Assess Pilots’ Cognitive State

The goal of the Crew State Monitoring (CSM) project is to use machine learning models trained with
physiological data to predict unsafe cognitive states in pilots such as Channelized Attention (CA) and
Startle/Surprise (SS). These models will be used in a real-time system that predicts a pilot's mental
state every second, a tool that can be used to help pilots recognize and recover from these mental states.

Pilots wore different sensors that collected physiological data such as a 20-channel electroencephalography (EEG), respiration, and galvanic skin response (GSR). Pilots performed non-flight benchmark tasks designed to induce these states, and a flight simulation with "surprising" or "channelizing" events.

The team created a pipeline to generate pilot-dependent models that trains on benchmark data, tune on a portion of a flight task, and be deployed onto the remaining flight task. The model is a series of anomaly-detection based ensembles, where each ensemble focuses on predicting a single state. Ensembles were comprised of several anomaly detectors such as One Class SVMs, each focusing on a different subset of sensor data.

We will discuss the performance of these models, as well as the ongoing research generalizing models
across pilots and improving accuracy.