Advanced Multimodal Solutions for Information Presentation

High-workload, fast-paced, and degraded sensory environments are the likeliest candidates to benefit from multimodal information presentation. For example, during extra-vehicular activity (EVA) and telerobotic operations, the sensory restrictions associated with such a hostile environment provide a major challenge to maintaining the situation awareness (SA) required for safe operations. In particular, orientation, navigation, and collision avoidance are critical aspects of EVA tasks that need to be addressed to ensure the safety of the crew and the success of the mission. Multimodal displays hold promise to enhance situation awareness and task performance by utilizing different sensory modalities and maximizing their effectiveness based on appropriate interaction between modalities.
Multimodal displays will also play an important role for long-duration information systems and will likely begin to be developed in the early phases of cislunar Gateway, and later lunar or Mars transit missions. Information systems are envisioned for LDEMs that require spacecraft with greater crew autonomy and increased dependence on computer-provided information needed to perform routine tasks, as well as time- and safety critical tasks. Such a system will require a single, common interface that is easy to learn and use and accesses key information from all relevant vehicle/habitat systems to enable task performance in both nominal and emergency conditions. Understanding of multimodal display technologies and their interactions will help to inform interface guidelines for LDEMs.
The scope of the current report is an analysis of potential multimodal display technologies for long duration missions and, in particular, will focus on their potential role in EVA activities. The review will address multimodal (combined visual, auditory and/or tactile) displays investigated by the National Aeronautics and Space Administration (NASA), industry, and Department of Defense (DoD). It also considers the need for adaptive information systems to accommodate a variety of operational contexts such as crew status (e.g., fatigue, workload level) and task environment (e.g., EVA, habitat, rover, spacecraft). Current approaches to guidelines and best practices for combining modalities for the most effective information displays are also reviewed. Potential issues in developing interface guidelines for LDEMs are briefly considered.