Trust in Collaborative Automation in High Stakes Software Engineering Work

The amount of autonomy in software engineering tools is increasing as developers build increasingly complex systems. Research in other domains shows that too much or too little trust in autonomous tools can have negative consequences, but we are not aware of any study that has investigated trust in autonomous tools in the highly interactive context of a software engineering workplace. We present the results of a ten week ethnographic case study of engineers collaborating with autonomous tools to write flight software at a large national space exploration organization to support high stakes missions. We find that trust in an autonomous software engineering tool in this setting was influenced by four main factors: the tool’s transparency, social context, an organization’s associated processes, and its usability. We outline theoretical implications for future research into trust in autonomous software engineering tools, and practical implications for tool designers and organizations conducting high stakes work with autonomous tools.