Measuring the Effectiveness of Human Autonomy Teaming

Human-Automation Teaming (HAT), is now recognized as a promising solution to the problems of humans managing increasingly complex work systems. A human-automation team can be defined as the interdependent coupling between one or more human operators and one or more autonomous systems requiring collaboration and coordination to accomplish system and task goals (e.g., Langan-Fox et al., 2009). In this conception, automated agents are considered team members that can operate at various levels of automation, be focused on one or more human-information-processing stages, and the interactions with human operators may be adaptable, adjustable or mixed initiative. We investigated some metrics for assessing HAT effectiveness in a demonstration of a HAT tool used by ground station operators in a Reduced Crew Operations project that was conducted at NASA Ames Human Automation Teaming Laboratory. In this paper, we focus on operator metrics of HAT effectiveness, specifically workload and operator behaviors.