Electronic checklists - Evaluation of two levels of automation

Two versions of an electronic checklist have been implemented in the Advanced Concepts Flight Simulator (ACFS) at NASA Research Center. The two designs differ in the degree of pilot involvement in conducting the checklists. One version (manual-sensed), requires the crew to manually acknowledge the completion of each checklist item. The other version (automatic-sensed), automatically indicates completed items without requiring pilot acknowledgement. These two designs and a paper checklist (as a control condition) were evaluated in line oriented simulation. Twelve aircrews from one major air-carrier flew a routine, four leg, short-haul trip. This paper presents and discusses the portion of the experiment that was concerned with measuring the effect of the degree of automation on the crews' performance. It discusses and presents evidence for a potential down-side of implementing an electronic checklist that is designed to provide fully redundant monitoring of human procedure execution and monitoring.