Enhanced Lighting Techniques and Augmented Reality to Improve Human Task Performance

One of the most versatile tools designed for use on the International Space Station (ISS) is the Special Purpose Dexterous Manipulator (SPDM) robot. Operators for this system are trained at NASA Johnson Space Center (JSC) using a robotic simulator, the Dexterous Manipulator Trainer (DMT), which performs most SPDM functions under normal static Earth gravitational forces. The SPDM is controlled from a standard Robotic Workstation. A key feature of the SPDM and DMT is the Force/Moment Accommodation (FMA) system, which limits the contact forces and moments acting on the robot components, on its payload, an Orbital Replaceable Unit (ORU), and on the receptacle for the ORU. The FMA system helps to automatically alleviate any binding of the ORU as it is inserted or withdrawn from a receptacle, but it is limited in its correction capability. A successful ORU insertion generally requires that the reference axes of the ORU and receptacle be aligned to within approximately 0.25 inch and 0.5 degree of nominal values. The only guides available for the operator to achieve these alignment tolerances are views from any available video cameras. No special registration markings are provided on the ORU or receptacle, so the operator must use their intrinsic features in the video display to perform the pre-insertion alignment task. Since optimum camera views may not be available, and dynamic orbital lighting conditions may limit viewing periods, long times are anticipated for performing some ORU insertion or extraction operations. This study explored the feasibility of using augmented reality (AR) to assist with SPDM operations. Geometric graphical symbols were overlaid on the end effector (EE) camera view to afford cues to assist the operator in attaining adequate pre-insertion ORU alignment.