Performance consequences of automation-induced 'complacency'

The effect of variations in the reliability of an automated monitoring system on human operator detection of automation failures was examined in two experiments. For four 30-min sessions, 40 subjects performed an IBM PC-based flight simulation that included manual tracking and fuel-management tasks, as well as a system-monitoring task that was under automation control. Automation reliability - the percentage of system malfunctions detected by the automation routine - either remained constant at a low or high level over time or alternated every 10 min from low to high. Operator detection of automation failures was substantially worse for constant-reliability than for variable-reliability automation after about 20 min under automation control, indicating that the former condition induced 'complacency'. When system monitoring was the only task, detection was very efficient and was unaffected by variations in automation reliability. The results provide the first empirical evidence of the performance consequences of automation-induced 'complacency'. We relate findings to operator attitudes toward automation and discuss implications for cockpit automation design.