Impact Real World System Validation

Introduction

NASA has developed a new evidence-based data-driven probabilistic risk assessment and tradespace analysis tool as a successor to the Integrated Medical Model. This updated decision support tool is known as IMPACT (Informing Mission Planning via Analysis of Complex Tradespaces). IMPACT estimates the frequency and consequences of medical conditions that might arise during exploration missions. A validation analysis of IMPACT was performed with respect to a set of International Space Station (ISS) and Shuttle Transportation System (STS) real world system (RWS) referent data due to the limited referent data available from exploration missions.

Methods

Observed mission and crew characteristics from STS and ISS missions were used as model inputs within MEDPRAT (Medical Extensible Dynamic Probabilistic Risk Assessment Tool). For each mission, two hundred thousand simulations were generated. For each mission, model outputs included occurrence counts for each condition, total medical events (TME), and the probability of loss of crew life (LOCL). These simulated model outputs were compared to the RWS referent data.

Results

The predicted number of total medical events exceeded the total RWS medical events for ISS missions and combined ISS and STS missions and fell within the 90% confidence interval for STS missions. For the 32 ISS missions simulated by IMPACT, the number of total medical events was overpredicted for 19 missions and fell within the 90% confidence interval for 13 missions. For the 21 STS missions, the total number of medical events was overpredicted for 3 missions, fell within the 90% confidence interval for 16 missions, and was underpredicted for 2 missions. Combined, 29 missions were in range, 22 were overpredicted, and 2 were underpredicted. The predicted LOCL probability for the 32 ISS missions, the 21 STS missions, and the combined ISS and STS missions was consistent with the zero LOCL events observed in the RWS referent data. The validation analysis included a comparison of the number of medical events predicted by IMPACT and the number of medical events observed in the RWS data on a condition-by-condition basis. For ISS missions, 50 conditions were in range, 52 conditions were statistically underpowered (not enough observed sample to draw any conclusions on precision), 8 conditions were overpredicted, and 9 conditions were underpredicted. Overall, only 14% (17/119) of conditions were out of range for STS missions, 40 conditions were in range, 59 conditions were statistically underpowered, 10 conditions were overpredicted, and 10 conditions were underpredicted. Overall, only 17% (20/119) of conditions were out of range. For combined ISS and STS missions, 11 conditions were overpredicted, and 11 conditions were underpredicted. Overall, only 18% (22/119) of conditions were out of range. For combined ISS and STS missions, 49 conditions were in range, 46 conditions were statistically underpowered, 18 conditions were overpredicted, and 8 conditions were underpredicted. Overall, 21% (26/121) of conditions were out of range.

Conclusion

The results of this validation analysis should not be interpreted as a pass/fail test of the validity of IMPACT. Instead, this validation analysis should be used to assess some of the IMPACT outcomes in terms of consistencies and inconsistencies with the ISS and STS RWS referent data.