Evaluation of a Remote Data Collection Method to Study Human-Automation Interaction and Workload

Technological advances have increased the automation of Uncrewed Aerial Vehicles, allowing human operators to manage multiple vehicles at a high-level without the need to understand low-level system behaviors. Previous laboratory studies have explored the relationship between reliability, trust, use of automation and the effects of number of vehicles under supervision on subjective workload. Due to the limitations resulting from the COVID-19 pandemic, in-person laboratory studies are not always possible. Therefore, this work aimed to investigate if remote data collection alternatives, such as Amazon’s Mechanical Turk, can provide comparative results as those obtained in laboratory settings. A study was conducted in the context of small drone operations. As expected, higher reliability led to higher trust ratings and the inclusion of more vehicles led to higher workload. In contrast, reliability unexpectedly had no effect on intention to use the automation. Though these results were encouraging, several limitations were identified.