NASA Logo

NTRS

NTRS - NASA Technical Reports Server

Back to Results
Effects of Transparency on Pilot Trust and Agreement in the Autonomous Constrained Flight PlannerWe performed a human-in-the-loop study to explore the role of transparency in engendering trust and reliance within highly automated systems. Specifically, we examined how transparency impacts trust in and reliance upon the Autonomous Constrained Flight Planner (ACFP), a critical automated system being developed as part of NASA's Reduced Crew Operations (RCO) Concept. The ACFP is designed to provide an enhanced ground operator, termed a super dispatcher, with recommended diversions for aircraft when their primary destinations are unavailable. In the current study, 12 commercial transport rated pilots who played the role of super dispatchers were given six time-pressured all land scenarios where they needed to use the ACFP to determine diversions for multiple aircraft. Two factors were manipulated. The primary factor was level of transparency. In low transparency scenarios the pilots were given a recommended airport and runway, plus basic information about the weather conditions, the aircraft types, and the airport and runway characteristics at that and other airports. In moderate transparency scenarios the pilots were also given a risk evaluation for the recommended airport, and for the other airports if they requested it. In the high transparency scenario additional information including the reasoning for the risk evaluations was made available to the pilots. The secondary factor was level of risk, either high or low. For high-risk aircraft, all potential diversions were rated as highly risky, with the ACFP giving the best option for a bad situation. For low-risk aircraft the ACFP found only low-risk options for the pilot. Both subjective and objective measures were collected, including rated trust, whether the pilots checked the validity of the automation recommendation, and whether the pilots eventually flew to the recommended diversion airport. Key results show that: 1) Pilots trust increased with higher levels of transparency, 2) Pilots were more likely to verify ACFPs recommendations with low levels of transparency and when risk was high, 3) Pilots were more likely to explore other options from the ACFP in low transparency conditions and when risk was high, and 4) Pilots decision to accept or reject ACFPs recommendations increased as a function of the transparency in the explanation. The finding that higher levels of transparency was coupled with higher levels of trust, a lower need to verify other options, and higher levels of agreement with ACFP recommendations, confirms the importance of transparency in aiding reliance on automated recommendations. Additional analyses of qualitative data gathered from subjects through surveys and during debriefing interviews also provided the basis for new design recommendations for the ACFP.
Document ID
20160011548
Acquisition Source
Ames Research Center
Document Type
Conference Paper
Authors
Sadler, Garrett
(ASRC Research and Technology Solutions, LLC Moffett Field, CA, United States)
Battiste, Henri
(San Jose State Univ. Moffett Field, CA, United States)
Ho, Nhut
(California State Univ. Northridge, CA, United States)
Hoffmann, Lauren
(NVH Human Systems Integration Canoga Park, CA, United States)
Lyons, Joseph
(Air Force Research Lab. Wright-Patterson AFB, OH, United States)
Johnson, Walter
(NASA Ames Research Center Moffett Field, CA United States)
Shively, Robert
(NASA Ames Research Center Moffett Field, CA United States)
Smith, David
(NASA Ames Research Center Moffett Field, CA United States)
Date Acquired
September 29, 2016
Publication Date
September 25, 2016
Subject Category
Air Transportation And Safety
Report/Patent Number
ARC-E-DAA-TN35072
Meeting Information
Meeting: IEEE/AIAA Digital Avionics Systems Conference (DASC)
Location: Sacramento, CA
Country: United States
Start Date: September 25, 2016
End Date: September 29, 2016
Sponsors: Institute of Electrical and Electronics Engineers, American Inst. of Aeronautics and Astronautics
Funding Number(s)
WBS: WBS 411931.02.61.01.08
CONTRACT_GRANT: NNA13AB88C
CONTRACT_GRANT: NNX12AB08A
Distribution Limits
Public
Copyright
Public Use Permitted.
Keywords
trust
automation
transparency
No Preview Available