SP-20205003605 TRA BP Guide Final [SP-20205003605] Technology Readiness Assessment Best Practices Guide National Aeronautics and Space Administration Office of the Chief Technologist [SP-20205003605] i Table of Contents 1. Scope and Applicability ............................................................................................. 1 2. Technology Readiness Level Definitions................................................................. 2 2.1. Progression of TRLs ........................................................................................................ 3 2.2. Terminology ..................................................................................................................... 4 2.2.1. Hardware Levels ......................................................................................................... 5 2.2.2. Configuration ............................................................................................................... 6 2.2.3. Environments .............................................................................................................. 6 2.3. TRL Formal Definition and Decomposition ................................................................... 6 2.3.1. Definitions of Technology Readiness Levels .............................................................. 6 2.3.2. TRL Decomposition .................................................................................................. 11 2.4. Fidelity of Analysis ........................................................................................................ 13 2.5. Fidelity of Build .............................................................................................................. 14 2.6. TRL and Lifetime Requirements ................................................................................... 15 3. Technology Readiness Assessment ...................................................................... 17 3.1. Technology Readiness Assessment Guidelines ........................................................ 17 3.1.1. Identification of Assessment Requirements .............................................................. 17 3.1.2. Identification of New Technology and Critical Technology Elements ....................... 18 3.1.3. Identification of Level of Integration for Test based on System Architecture ............ 19 3.1.4. Technology Readiness Assessment ......................................................................... 20 3.1.5. Roll up of TRL ........................................................................................................... 26 3.2. Technology Readiness Assessment Process ............................................................. 26 3.2.1 Convening a TRA ....................................................................................................... 26 3.2.2. Relationship of TRAs to Project Life Cycle ............................................................... 27 3.2.3. Technology Readiness Assessment Report ............................................................. 28 4. Risk Assessment Associated with Progression of Maturity ................................ 30 5. Nomenclature/Acronyms ......................................................................................... 31 Appendix A. Center TRA Processes ........................................................................... 33 [SP-20205003605] 1 1. Scope and Applicability The use of new technology in NASA flight projects enables new scientific results and technical capabilities otherwise unachievable. However, transitioning new technology to flight elements is difficult and may introduce significant risks. Similar risks exist in the general case of research and technology development projects. The projects face the additional challenge of uncertainty in the time and context of the future infusion path for a particular technology. A balance between benefit and risk is necessary to progress space missions. A firm grasp of the risks balanced against benefits supports an environment where innovation is nurtured, rather than avoided. Realistic and consistent assessments throughout the technology development lifecycle lead to greater infusion of technology into missions with a concomitant cost realism for development. This guide establishes standard definitions and best practices for conducting technology readiness assessments (TRAs) for in flight projects and NASA’s research and technology missions. These best practices originated from a NASA TRA Committee that met weekly in 2014. The team was co-led by the Headquarters (HQ) Office of the Chief Engineer (OCE) and the HQ Office of the Chief Technologist (OCT), and included one to two representatives from each Center. This best practices guide is suitable for reliably determining a meaningful technology readiness level (TRL) to both technology development and flight development projects.1 This guide defines TRLs and shares best practices for TRAs, including process and implementation. This guide also suggests a process for assessing risk associated with technology maturing to a higher level. The guide includes an Appendix that shares examples of TRA processes used in different Centers including Goddard Space Flight Center (GSFC), Ames Research Center (ARC), and Jet Propulsion Laboratory (JPL). 1 This guidebook addresses technology at the system, subsystem, and assembly level. Software is integral and applicable to systems, but this guide does not address the maturity of software itself. Software development guidelines and processes can be found in NASA Procedural Requirement (NPR)- 7150.2 NASA Software Engineering Requirements. [SP-20205003605] 2 2. Technology Readiness Level Definitions “Technology Readiness Level” is a scale for measuring the maturity of a technology, defined in NPR 7123.1 and NASA/SP-2007-6105. The TRL describes the performance history of a given system, subsystem, or component relative to a set of levels first described at NASA HQ in the 1980s. The TRL describes the state of a given technology and provides a baseline from which maturity is gauged and advancement defined. As a set of metrics, TRLs enable the standardized assessment of the maturity of a particular technology and the consistent comparison of the maturity between different types of technology in the context of a specific application, implementation, and operational environment. TRLs range from 1 - Basic Technology Research to 9 - Systems Test, Launch, and Operations. Typically, a TRL of 6 (technology demonstrated in a relevant environment) or higher is required for a technology to be integrated into a flight system. Figure 2-1 provides a high-level illustration of the TRL level scale, using the “thermometer scale” as a metaphor for increasing technology maturity starting with basic research and progressing through flight system operation. Figure 2-1: Thermometer Scale for NASA’s Technology Readiness Levels The remainder of this section describes the progression of TRLs, terminology associated with TRLs, and official definitions for each TRL. This section also provides guidelines for evaluating the fidelity of the analysis and the fidelity of build used for the [SP-20205003605] 3 new technology under assessment. Last, the section provides requirements, by TRL, for assessing the lifetime of a technology. 2.1. Progression of TRLs TRL measures the progression of new technology from concept to use in an operational flight mission. The new technology conception occurs starting at TRL 1 through TRL 3. Development and demonstration occur between TRLs 4 through TRL 6. Once TRL 6 is demonstrated, the risk associated with the new technology is roughly equivalent to the risk of a new design that employs standard engineering practice and is bounded by previously implemented ground-based systems. NASA practice recommends technology demonstrates TRL 6 prior to the Preliminary Design Review (PDR). Following TRL 6 demonstration, the standard engineering development cycle for new designs occurs. This cycle includes building and testing an engineering unit, detailed analysis, and detailed drawings prior to the Critical Design Review (CDR). The design is flight qualified at the subsystem and system level by the ground systems or flight project leading to flight readiness at TRL 8. Successful operation of ground systems supporting launch and in flight constitutes TRL 9 completion. In some cases, it is desirable to demonstrate a new technology in independent flight prior to incorporation in the flight program. This spaceflight technology demonstration constitutes TRL 7. The technology demonstration is a representative high fidelity prototype that is demonstrated in flight, but not necessarily a “build-to-print” unit that might be used on a specific future operational space flight mission. For ground support systems, it is desirable in some cases to demonstrate a new technology in a parallel "shadow mode" with current operations to demonstrate performance prior to incorporation supporting launch. [SP-20205003605] 4 Figure 2.1-1 Two paths to flight. The most common path progresses from TRL 1 to TRL 6, skips TRL 7, and then progresses through TRL 8 to 9. For new technologies where demonstration in space is critical to risk reduction, TRL 7 is also included. Figure 2.1-2 Configuration Fidelity and Environmental Spectrum 2.2. Terminology The first step in developing a uniform TRL assessment is to define terms. It is crucial to develop and use a consistent set of definitions over the course of the program/project. [SP-20205003605] 5 2.2.1. Hardware Levels Proof of Concept: Analytical and experimental demonstration of hardware/software concepts that may or may not be incorporated into subsequent development and/or operational units. Breadboard: A low fidelity unit that demonstrates function only, without respect to form or fit. It often uses commercial and/or ad hoc components and is not intended to provide definitive information regarding operational performance. Brassboard: A medium fidelity functional unit that typically tries to make use of as much of the final product as possible and begins to address scaling issues associated with the operational system. It does not have the engineering pedigree in all aspects but is structured to be able to operate in simulated operational environments in order to assess performance of critical functions. Prototype Unit: The prototype unit demonstrates form and function at a scale deemed to be representative of the final product operating in its operational environment. A subscale test article provides fidelity sufficient to permit validation of analytical models capable of predicting the behavior of full-scale systems in an operational environment. Engineering Unit: A high fidelity unit that demonstrates critical aspects of the engineering processes involved in the development of the operational unit. Engineering test units are intended to closely resemble the final product (hardware/software) to the maximum extent possible. They are built and tested to establish confidence that the design will function in the expected environments. In some cases, the engineering unit will become the final product, assuming proper traceability has been exercised over the components and hardware handling. Protoflight Unit: Protoflight units are developed for cases when a qualification unit is not developed (due to cost or schedule constraints). The protoflight unit is intended for flight or deployment and operations. A limited set of qualification and tests are performed on the protoflight to preserve its ability to function and life expectancy. Full acceptance testing is performed. Flight Qualification Unit: Flight hardware that is tested to the levels that demonstrate the desired qualification level margins. Sometimes this means testing to failure. This unit is never used operationally. Flight Unit: The flight unit is the end item unit intended for flight. It is subjected to formal acceptance testing. Flight Spare: The flight spare is the spare end item for flight. It is subjected to formal acceptance testing. It is identical to the flight unit. [SP-20205003605] 6 2.2.2. Configuration Mission Configuration: The final architecture/system design of the product that will be used in the operational environment. If the product is a subsystem/component, then it is embedded in the actual system in the actual configuration used in operation. 2.2.3. Environments Laboratory Environment: Tests in a laboratory environment are for the purpose of demonstrating the underlying principles of technical performance/functionality without respect to the impact of environment. A laboratory or field environment is not required to address the environment to be encountered by the system, subsystem, or component during its intended operation. Relevant Environment: A relevant environment approximates a specific subset of the operational environment and focuses specifically on “stressing” the technology advancement in question. Not all systems, subsystems, and/or components need to be operated in the operational environment in order to satisfactorily address performance margin requirements. Consequently, the relevant environment is the specific subset of the operational environment that is simulated in ground test facilities required to demonstrate critical “at risk” aspects of the final product performance in an operational environment. Flight Qualification Environment: A flight qualification environment is simulated in ground test facilities that the flight project defines will verify the system with margin. Operational Environment: The operational environment is where the final product will be operated. In the case of space flight equipment, it is the space or planetary environment. In the case of ground support equipment supporting the launch vehicle and spacecraft, it is the launch operational environment. 2.3. TRL Formal Definition and Decomposition 2.3.1. Definitions of Technology Readiness Levels Table 2.3.1.-1: Definitions of Technology Readiness Levels TRL Definition Description Success criteria 1 Basic principles observed and reported. Scientific knowledge generated underpinning technology concepts/applications. Peer reviewed documentation of research underlying the proposed concept/application. [SP-20205003605] 7 Examples: a. Initial Paper published providing representative examples of phenomenon as well as supporting equations for a concept. b. Conference presentations on concepts and basic observations presented within the scientific community. 2 Technology concept and/or application formulated. Invention begins, practical application is identified but is speculative, no experimental proof or detailed analysis is available to support the conjecture. Documented description of the application/concept that addresses feasibility and benefit. Examples: • Carbon nanotube composites were created for lightweight, high-strength structural materials for space structures • Mini-CO2 Scrubber: Applies advanced processes to remove carbon dioxide and potentially other undesirable gases from spacecraft cabin air. 3 Analytical and experimental proof- of-concept of critical function and/or characteristics. Research and development are initiated, including analytical and laboratory studies to validate predictions regarding the technology. Documented analytical/experimental results validating predictions of key parameters. Examples: a. High efficiency Gallium Arsenide solar panels for space application is conceived for use over a wide temperature range. The concept critically relies on improved welding technology for the cell assembly. Samples of solar cell assemblies are manufactured and submitted to a preliminary thermal environment test at ambient pressure for demonstrating the concept viability. b. A fiber optic laser gyroscope is envisioned using optical fibers for the light propagation and Sagnac Effect. The overall concept is modeled including the laser source, the optical fiber loop, and the phase shift measurement. The laser injection in the optical fiber and the detection principles are supported by dedicated experiments. c. In Situ Resource Utilization: Demonstrated the application of a cryo-freezer for CO2 acquisition and microwave processor for water extraction from soils. 4 Component and/or breadboard validation in laboratory environment. A low fidelity system/component breadboard is built and operated to demonstrate basic functionality in a laboratory environment. Documented test performance demonstrating agreement with analytical predictions. Documented definition of potentially relevant environment. [SP-20205003605] 8 Examples: a. Fiber optic laser gyroscope: A breadboard model is built including the proposed laser diode, optical fiber and detection system. The angular velocity measurement performance is demonstrated in the laboratory for one axis rotation. b. Bi-liquid chemical propulsion engine: A breadboard of the engine is built and thrust performance is demonstrated at ambient pressure. Calculations are done to estimate the theoretical performance in the expected environment (e.g., pressure, temperature). c. A new fuzzy logic approach to avionics is validated in a lab environment by testing the algorithms in a partially computer-based, partially bench-top component (with fiber optic gyros) demonstration in a controls lab using simulated vehicle inputs. d. Variable Specific Impulse Magnetosphere Rocket (VASIMR): 100 kW magnetoplasma engine operated 10 hours cumulative (up to 3 minutes continuous) in laboratory vacuum chamber. 5 Component and/or brassboard validated in relevant environment. A medium-fidelity component and/or brassboard, with realistic support elements, is built and operated for validation in a relevant environment so as to demonstrate overall performance in critical areas. Documented test performance demonstrating agreement with analytical predictions. Documented definition of scaling requirements. Performance predictions are made for subsequent development phases. Examples: a. A 6.0-meter deployable space telescope comprised of multiple petals is proposed for near infrared astronomy operating at 30K. Optical performance of individual petals in a cold environment is a critical function and is driven by material selection. A series of 1m mirrors (corresponding to a single petal) were fabricated from different materials and tested at 30K to evaluate performance and to select the final material for the telescope. Performance was extrapolated to the full-sized mirror. b. For a launch vehicle, TRL 5 is the level demonstrating the availability of the technology at subscale level (e.g., the fuel management is a critical function for a re-ignitable upper stage). The demonstration of the management of the propellant is achieved on the ground at a subscale level. c. International Space Station (ISS) Additive Manufacturing Facility: Characterization tests compare parts and material properties of polymer specimens printed on ISS to copies printed on the ground. 6 System/sub-system model or prototype demonstration in a relevant environment. A high-fidelity prototype of the system/subsystems that adequately addresses all critical scaling issues is built and tested in a relevant environment to demonstrate performance under critical environmental conditions. Documented test performance demonstrating agreement with analytical predictions. [SP-20205003605] 9 Examples: a. A remote sensing camera includes a large 3-meter telescope, a detection assembly, a cooling cabin for the detector cooling, and an electronics control unit. All elements have been demonstrated at TRL 6 except for the mirror assembly and its optical performance in orbit, which is driven by the distance between the primary and secondary mirrors needing to be stable within a fraction of a micrometer. The corresponding critical part includes the two mirrors and their supporting structure. A full-scale prototype consisting of the two mirrors and the supporting structure is built and tested in the relevant environment (e.g., including thermo-elastic distortions and launch vibrations) for demonstrating the required stability can effectively be met with the proposed design. b. Vacuum Pressure Integrated Suit Test (VPIST): Demonstrated the integrated performance of the Orion suit loop when integrated with human-suited test subjects in a vacuum chamber. 7 System prototype demonstration in an operational environment. A high-fidelity prototype or engineering unit that adequately addresses all critical scaling issues is built and functions in the actual operational environment and platform (ground, airborne, or space). Documented test performance demonstrating agreement with analytical predictions. Examples: a. Mars Pathfinder Rover flight and operation on Mars as a technology demonstration for future micro-rovers based on that system design. • First flight test of a new launch vehicle, which is a performance demonstration in the operational environment. Design changes could follow as a result of the flight test. • In-space demonstration missions for technology (e.g., autonomous robotics and deep space atomic clock). Successful flight demonstration could result in use of the technology in a future operational mission • Robotic External Leak Locator (RELL): Originally flown as a technology demonstrator, the test article was subsequently put to use to help operators locate the likely spot where ammonia was leaking from the ISS External Active Thermal Control System Loop B. 8 Actual system completed and “flight qualified” through test and demonstration. The final product in its final configuration is successfully demonstrated through test and analysis for its intended operational environment and platform (ground, airborne, or space). If necessary*, life testing has been completed. Documented test performance verifying analytical predictions. [SP-20205003605] 10 Note: *“If necessary” refers to the need to life test either for worn out mechanisms, for temperature stability over time, and for performance over time in extreme environments. An evaluation on a case-by-case basis should be made to determine the system/systems that warrant life testing and the tests begun early in the technology development process to enable completion by TRL 8. It is preferable to have the technology life test initiated and completed at the earliest possible stage in development. Some components may require life testing on or after TRL 5. Examples: a. The level is reached when the final product is qualified for the operational environment through test and analysis. Examples are when Cassini and Galileo were qualified, but not yet flown. • Interim Cryo-Propulsion Stage (ICPS): A Delta Cryogenic Second Stage modified to meet Space Launch System requirements for Exploration Mission-1 (EM-1), qualified and accepted by NASA for flight on EM-1. 9 Actual system flight proven through successful mission operations. The final product is successfully operated in an actual mission. Documented mission operational results. Examples: a. Flown spacecraft (e.g., Cassini, Hubble Space telescope) b. Technologies flown in an operational environment c. Nanoracks CubeSat Deployer: Commercially developed and operated small satellite deployer onboard the ISS. Table 2.3.1-2 below outlines the advancement of a single technology through all nine TRLs. This example illustrates the timing and progression of the full maturation cycle, according to the definitions given above. Table 2.3.1-2: Technology Readiness Level Example – Terrain Relative Navigation TRL Year Achieved Technology State 1 1989 Mars pinpoint landing concepts and enabling technologies were explored under the Mars Rover Sample Return mission study (A. Klumpp, “Pinpoint landing concepts for the Mars Rover Sample Return mission”, AAS Paper 89-046, Annual Rocky Mountain Guidance and Control Conference, 1989). 2 2004 Formulated the concept of terrain relative navigation, its benefits, and desired performance characteristics for many solar system bodies. Responded to release of the NASA Research Announcement for the New Millennium Program Space Technology – 9 (ST-9) mission, with Appendix D on Terrain-Guided Automatic Landing System for Spacecraft (TGALS). 3 2005 Studies funded by Mars Technology program provided analytical and experimental proof-of-concept of onboard registration of features seen in descent imagery to Mars orbital imagery (Y. Cheng, “Landmark based position estimation for pinpoint landing on Mars”, IEEE International Conference on Robotics and Automation, 2005) [SP-20205003605] 11 4 2007 By the end of the Study Phase of the ST-9 mission, terrain relative navigation algorithms were tested by off-line processing of a set of IMU, descent image, and ground truth data collected during a sounding rocket flight conducted to emulate the conditions of Mars landing (A. Johnson, et al, “A general approach to terrain relative navigation for planetary landing,” AIAA Infotech@Aerospace Conference, 2007). Performance agreed with analytical predictions from planetary imagery and a simulation of Mars imagery. 5 2013 Using funding from the NASA SMD Mars technology Program, the real-time Lander Vision System (LVS) was designed and implemented on prototype computing hardware with a path to flight implementation. The compute element was interfaced to a COTS camera and IMU that met the requirements for Mars landing. The performance of the working system was demonstrated to meet processing time requirements in the lab. Short range lab test results scaled well to predicted performance at Mars EDL ranges. (A. Johnson et al., “Design and Ground Test Results for the Lander Vision System”, AAS GN&C Conference 2013). 6 2015 The prototype LVS implementation was completed and tested in real-time on a manned helicopter over a wide variety of scenes. (A. Johnson et al., “Real-Time Terrain Relative Navigation Test Results from a Relevant Environment for Mars Landing” AIAA SciTech Conference 2015). The LVS preliminary design for Mars 2020 was completed and reviewed at the Mars 2020 TRN PDR, which included extensive simulation results for Mars 2020 landing. 7 2015 The LVS prototype was integrated with a vertical take-off and vertical landing rocket and used successfully in two closed loop pin-point landing demonstrations (N. Trawny et al., “Flight testing of terrain-relative navigation and large-divert guidance on a VTVL rocket,” AIAA Space Conference 2015). 8 2020 Mars 2020 LVS implementation was completed, environmentally tested and delivered to spacecraft integration. (2018). Software and firmware completed (2019). Real-time LVS helicopter field test completed successfully and results match simulation (A. Johnson et al., “The Mars 2020 Lander Vision System Field Test, AAS GN&C Conference, 2020). All V&V completed including flight system testing (April 2020). 9 2021 The 2020 Mars rover mission achieved this milestone successfully by using TRN during terminal descent. 2.3.2. TRL Decomposition The factors that affect the TRL include performance/function, fidelity of the physical realization of the technology often referred to as “form and fit,” and survivability in the operational environment. These factors are measured against a set of requirements derived for the use of the technology in an applicable mission. Since part of technology development includes understanding the physical basis for the technology, completion criteria for each of the TRLs also depends on the fidelity of analyses that predict technology performance. Table 2.3.2-1 summarizes the TRL definitions broken down by [SP-20205003605] 12 these factors. Table 2.3.2-1: TRL Definition and Decomposition by Factor T R L Definition Completion Criteria Mission Req. Performance/ Function Fidelity of Analysis Fidelity of Build Level of Integration Environment Verification 1 Basic Principles observed and reported Peer reviewed documented principles Generic class of missions Knowledge underpinning technology concept/ applications Physical principles identified NA NA NA 2 Technology concept and/or application formulated Documented description that addresses feasibility and benefit Generic class of missions Concept formulated Feasibility presented NA NA NA 3 Analytical and/or experimental proof-of- concept of critical function. Documented analytical/ experimental results validating predictions of key parameters Generic class of missions Proof-of- Concept demonstrated analytically or experimentally Low fidelity: to predict key performance parameters NA, but could be low fidelity bread- board NA NA 4 Component and/or bread- board validated in laboratory environment Documented test performance demonstrating agreement with analytical predictions. Documented definition of relevant environment Generic class of missions Basic functionality/ performance demonstrated Medium fidelity: to predict key performance parameters and life limiting factors as a function of relevant environments Low fidelity: bread- board Component/ Assembly Tested in laboratory for critical environments. Relevant environments identified. Life- limiting mechanisms identified 5 Component and/or brassboard validated in relevant environment Documented test performance demonstrating agreement with analytical predictions. Documented definition of scaling requirements Generic or specific class of missions Basic functionality/ performance maintained Medium fidelity: to predict key performance parameters and life limiting factors as a function of relevant environments Medium fidelity: brassboard with realistic support elements Component/ Assembly Tested in relevant environments. Characterize physics of life- limiting mechanisms and failure modes 6 System/ subsystem Documented test Specific mission Required functionality/ Medium fidelity: to High fidelity: Subsystem/ System Tested in relevant [SP-20205003605] 13 Table 2.3.2-1: TRL Definition and Decomposition by Factor T R L Definition Completion Criteria Mission Req. Performance/ Function Fidelity of Analysis Fidelity of Build Level of Integration Environment Verification model or prototype demonstrated in a relevant environment performance demonstrating agreement with analytical predictions performance demonstrated predict key performance parameters and life limiting factors as a function of operational environments prototype that addresses all critical scaling issues environments. Verify by test that the technology is resilient to the effects of life- limiting mechanisms 7 System prototype demonstration in an operational environment Documented test performance demonstrating agreement with analytical predictions Technology demo. mission Required functionality/ performance demonstrated High fidelity: to predict key performance parameters and life limiting factors as a function of operational environments High Fidelity: prototype or engineerin g unit that addresses all critical scaling issues Subsystem/ System Tested in actual operational environment 8 Actual system completed and “flight qualified” through test and demonstration Documented test performance verifying requirements and analytical predictions Specific mission Required functionality/ performance demonstrated High fidelity: to predict key performance parameters and life limiting factors as a function of operational environments Final product: Flight unit; Life test unit for life limited items* System Tested in project environmental verification program. Completed life tests. 9 Actual system flight proven through successful mission operations Documented mission operational results verifying requirements Specific mission Required functionality/ performance demonstrated High fidelity: to predict key performance parameters and life limiting factors as a function of operational environments Final product: Flight unit System Operated in actual operational environment 2.4. Fidelity of Analysis Analysis and the development of analytical models for new technology is important for predicting performance during tests, understanding margins, conducting trades, assessing risks for “Test as You Fly” exceptions, as part of test beds, and many other reasons. Analysis is a key part of the Completion Criteria for each TRL (NPR 7123.1). [SP-20205003605] 14 The fidelity of the analysis is assessed against three aspects—its content, its basis, and its validity. Characteristics associated with these are given in Table 2.4.-1 Table 2.4.-1: Fidelity of Analysis Fidelity Content Basis Validation Low Key performance parameters (KPPs). Includes critical parts. Quantitative relationship between KPPs to predict values at one design point. May be based on “rules of thumb” and empirical knowledge. NA Mediu m Key performance parameters and life limiting factors. Includes critical parts and interfaces. Quantitative relationship between KPPs and life limiting factors to predict values as a function of relevant environments. Based on analytical physical principles and “first order” equations. Validation against test of technology with moderate level of Model Uncertainty Factor (MUF) assessed. Range of applicability and limitations identified and understood. High Near complete set of parameters including key performance parameters, life limiting factors, and other relevant parameters. Includes near complete set of parts and interfaces. Quantitative relationship between KPPs and life limiting factors with additional level of detail to predict values as a function of operational environments. Based on analytical physical principles, equations, and statistical methods. Use of high fidelity modeling tools such as finite element analysis structural, and thermal codes and detailed optical codes. Validation against test and other analytical models with low level of MUF assessed. Range of applicability and limitations identified and understood. 2.5. Fidelity of Build For new technology development, the fidelity of the physical realization progresses from low fidelity breadboards to medium fidelity brassboards to high fidelity prototypes. Once the new technology has been demonstrated as a prototype at the subsystem or system level it can be treated using the “standard engineering” approach for a new design with an engineering unit followed by qualification unit and a flight unit. If the protoflight approach to qualification is used, only the protoflight unit is needed. The “standard engineering” development cycle uses engineering design tools to produce the preliminary design for PDR. The detailed design, completed by CDR, includes detailed analysis, drawings or analytical models, and, for new designs, an engineering unit that is tested over the range of relevant environments. For designs that incorporate new technology an additional step is added to the design process, prior to PDR, culminating with a prototype tested in relevant environments that demonstrates TRL 6. The objective of this additional design effort is to bring the new technology to a similar level of maturity as the “standard engineering” elements. Once TRL 6 is achieved the “standard engineering” design approach is followed. [SP-20205003605] 15 Fidelity characteristics of build units are given in Table 2.5.-1. Table 2.5.-1: Fidelity of Build Unit Purpose Performance/ Function Form and Fit/ Scaling Environmental Requirements Pedigree N ew T ec hn ol og y D ev el op m en t Breadboard Proof-of- concept for a potential design Demonstrate performance/ function Not required, e.g. laid out flat on lab table Tested in a laboratory environment NA Brassboard Demonstrate feasibility of form and fit, environments Demonstrate performance/ function Approximate (not flat) with scaling factors understood Designed to meet relevant environmental requirements NA Prototype Representative design; pathfinder; demonstrator Tested to meet performance/ function requirements Representative with scaling factors understood Tested to meet relevant environmental requirements NA, but may be partial or full E ng in ee ri ng D ev el op m en t Engineering Unit Finalize detailed design Tested to meet performance/ function requirements Exact as known at time of build Tested to meet relevant environmental requirements NA, but may be partial or full Qualification Unit Qualify design Tested to meet performance/ function requirements Exact as known at time of build Tested to meet flight qualification environmental requirements Full Flight Unit Final Product Tested to meet performance/ function requirements Exact Tested to meet flight qualification environmental requirements Full Flight Spare Final Product Tested to meet performance/ function requirements Exact Tested to meet flight qualification environmental requirements Full 2.6. TRL and Lifetime Requirements For technologies where lifetime is a major consideration and a key technology issue, lifetime needs to be addressed as part of the technology readiness assessment. We recommend that technology maturation programs address life requirements as follows: [SP-20205003605] 16 • TRL 4 – Identify life-limiting mechanisms and failure modes. • TRL 5 – Characterize, by means of test, the physics of the life-limiting mechanisms and failure modes, or develop and validate an analytical model/simulation that predicts life limiting mechanisms and failure modes. • TRL 6 – Verify by test or analysis that the technology is resilient to the effects of life-limiting mechanisms. • TRL 8 – Complete life tests. [SP-20205003605] 17 3. Technology Readiness Assessment This section describes the guidelines and process for conducting a TRA. The guidelines provide the steps of a TRA, and the process describes the best practices for implementing those steps. 3.1. Technology Readiness Assessment Guidelines The TRA approach has five steps: 1. Identify the performance/functionality and environmental requirements against which the TRL will be assessed. 2. Identify the new technology elements. 3. Identify the level of integration or configuration in which the technology readiness needs to be tested. 4. Conduct the TRA of each element. 5. Roll-up the TRA to higher levels of integration. In addition, an assessment of risk may be conducted if desired. Each of these steps is discussed in more detail below. 3.1.1. Identification of Assessment Requirements TRL assessment is performed against a specific set of requirements. A new technology can be at a different TRL depending on the requirements. For instance, a technology demonstrated in low Earth orbit is TRL 9 for an Earth remote-sensing mission. However, this same technology may be TRL 4 for a mission to land on the surface of Venus. Therefore, the mission requirements need to be identified and agreed upon by the technology provider and the customer. The readiness of each technology, even if it has been flown, needs to be assessed when used in a different environment. Even in a known environment, new technologies replacing an obsolete, previously TRL 9 technology, still need a TRA. For lower level TRLs 1 - 3, details of the requirements may not be available. However, it is still critical to identify assumptions about the performance and environments that are applicable to the new technology. [SP-20205003605] 18 Figure 3.1.1-1: Flow chart to determine which of 3 categories—new technology, standard engineering, or heritage—an element is assigned to. Note that this flow chart does not identify TRL. 3.1.2. Identification of New Technology and Critical Technology Elements Often only “new technology” elements are assessed for technology readiness as in the case of many NASA Announcements of Opportunity or demonstration of TRL 6 by PDR. Not all new designs are necessarily new technology. Some may be considered “standard engineering.” The flow chart in Figure 3.1.1-1 is provided to place the elements of a flight system into one of three categories: “new technology,” “standard engineering,” or “heritage.” The flow chart determines whether or not the characteristic of the element is new or novel, bounded by demonstrated capability on the ground, or demonstrated in flight operations. The chart provides a simpler, systematic way to identify the categories. For “standard engineering,” demonstration is accomplished by test or validated analysis. A successful test is desirable and definitive. Demonstration may also include design by means of a validated, high fidelity analytical model based on measured physical parameters. For example, the materials and properties of a structural design are tested and characterized over a specific temperature range. Therefore, this structural design would be demonstrated over that temperature range. In this case, the structure itself was not tested over the temperature range. Should the structure be exposed to an extreme environment, such as the surface of Venus, for which its material properties had not been characterized, it would fall into the “new technology” bin. The technology developer and the customer of the technology need to agree upon the identification of “new technology” elements. A critical technology element (CTE) is defined as a new technology that is required for an operational mission. OCE/OCT Definition Sub-team -3 New Technology Engineering Heritage Performance or func3on new or novel? Yes No Performance or func3on bounded by demonstrated capability? Yes No Form and fit bounded by demonstrated capability? Yes No Performance or func3on demonstrated opera3onally? Yes No Form and fit demonstrated opera3onally? Yes No Environment less adverse than that demonstrated opera3onally? Yes No No process change in manufacture ? Yes No Interface within accep3ng system capability? Yes No Environments bounded by demonstrated capability? Yes No Start [SP-20205003605] 19 3.1.3. Identification of Level of Integration for Test based on System Architecture The configuration for TRL verification occurs at the lowest level of integration that exhibits the new performance/functionality and for which the interfaces remain in the “standard engineering” realm. Figure 3.1.3-1 illustrates a hierarchical breakdown of a system into its levels of integration. This breakdown includes subsystems, consisting of assemblies that in turn consist of lower level elements. In this example the elements determined to be “new technology” are colored red, while those “standard engineering” are colored grey. Element A and B are “new technologies” that when combined into Assembly A provide the new capability. Figure 3.1.3-1: Hierarchical breakdown of the system structure Another factor is the maturity of the interfaces (mechanical, thermal, electrical, data, etc.). Interfaces can be characterized in a manner similar to elements. If the new technology is a “drop in” replacement for the element it is replacing, the interface is “heritage.” No change is required to the interface. If the interface requirements are within the bounds of previously demonstrated interfaces, it is “standard engineering.” An example might be that the data throughput is 10 kbps, easily achievable with standard protocols. However, if the data throughput were 10 tbps, it is outside the bounds of demonstrated performance, and then the interface, itself, requires “new technology” development. TRL 4 and 5 can be demonstrated at the assembly level and do not necessarily address the interaction with other elements of the system. TRL 6, however, is demonstrated at the lowest level of integration for which new behavior can be tested and the interface is “standard.” In the example in Figure 3.1.3-1 this corresponds to Assembly A. Once TRL 6 is demonstrated for Assembly A, then the whole system is also at TRL 6 because the higher level of integration has no challenging interfaces. [SP-20205003605] 20 3.1.4. Technology Readiness Assessment The goal of a TRA is to increase technology infusion by conducting realistic and consistent evaluations throughout the technology lifecycle. Technology infusion is the pathway by which technologies, previously unimplemented, move from their current status into mission use. In addition to conducting TRAs, pursuing multiple pathways helps advance infusion and mitigate risk. Both NASA Research Centers and Mission Centers conduct TRAs to advance technology infusion. TRAs at TRL 6 provide a forum for the project and technologist to agree on the readiness of technology for the mission, understand the risks, and plan for maturation of the technology. The TRA is accomplished by determining the TRL assessment for each element using a set of questions for each level. These questions are answered with objective evidence that can be a test report, a signed document, an analytical result, etc. All questions must be answered successfully to demonstrate the given TRL level. The customer, i.e. the party paying for the development, likely varies as the technology progresses from low TRL to higher TRL. At lower TRL the customer may be a technology development program such as the NASA Game Changing Development or Picasso programs, or for a directed project it may be the future flight project. The higher TRL development is largely supported by the operational flight project. The hand-off to an operational flight project typically occurs at TRL 5, TRL 6, and TRL 7. The TRA provides a basis for discussing the work to be completed for flight readiness. The questions are broken into four sections. Section A identifies agreements between the technology provider and the customer. These agreements identify the scope of the technology development. These questions help identify and articulate assumptions made prior to the development of requirements for early TRLs. For higher TRL, in some cases both design requirements and test requirements are identified. The design requirements are associated with a specific mission or a range of missions, but not all those requirements need be demonstrated by test to achieve a given TRL. This approach assures that the whole scope of the technology application is communicated. The technology should be capable of meeting the design requirements. Sections B and C address the work accomplished to achieve a certain TRL. Section B addresses the analysis results while Section C addresses the test results. These show the outcome of the technology development effort. Lastly, Section D addresses the data products that provide the objective evidence needed to document the agreements and results. These data products can range from a report or publication at lower TRLs to detailed design and performance data for higher TRLs. Questions for TRL 8 and 9 are not given since these are covered by operational flight project efforts. TRL 8 is demonstrated when a mission is ready for launch and TRL 9 when it is successfully operated. A flight product delivery review and certification of flight readiness cover TRL 8 and TRL 9 questions, respectively. Every flight project will have [SP-20205003605] 21 its own set of reviews. Questions: TRL 1 A. Agreement between technology deliverer and customer (here the customer is, for instance, the sponsor supporting the research). 1. What are the CTEs? 2. What are the benefits of the new technology? 3. What are the applications and the performance/function needed for those applications? 4. What analysis is needed? This includes, at minimum, the following. a. Basic principles and physical laws underpinning the technology concept 5. What data is used to capture the agreements and results? B. Analysis results: 6. Is the concept supported by basic principles? C. Test results: NA D. Data Products: 7. Are the data products, agreed to in Question 5, above, complete? (Here this could be a report to the sponsor or a scientific publication). Questions: TRL 2 A. Agreement between technology deliverer and customer: (Here the customer is, for instance, the sponsor supporting the research). 1. What are the CTEs? 2. What are the benefits of the new technology? 3. What are the applications and the performance/function needed for those applications? 4. What analysis is needed? This includes, at minimum, the following. a. Concept formulation b. Feasibility demonstration 5. What data are used to capture the agreements and results? B. Analysis results: 6. Is the concept shown to be feasible? C. Test results: NA D. Data Products: 7. Are the data products, agreed to in Question 5, above, complete? (Here this could be a report to the sponsor or a scientific publication.) Questions: TRL 3 A. Agreement between technology deliverer and customer: customer (here the customer is, for instance, the sponsor supporting the research). 1. What are the CTEs? 2. What are the benefits of the new technology? [SP-20205003605] 22 3. What are the applications and the performance/function needed for those applications? 4. What are the likely operating environments? 5. What are the analysis requirements? This includes, at a minimum, the following: a. Key performance parameters (KPPs) b. Relationship between KPPs based on empirical knowledge and “rules of thumb” 6. What are the analyses and/or experiments needed to provide a “proof-of- concept”? 7. What data are used to capture the agreements and results? B. Analysis results 8. Does the predicted performance for the key parameters provide the “proof-of-concept”? C. Test results (when applicable) 9. Is the “proof-of-concept” successfully demonstrated? D. Data Products: 10. Are the data products, agreed to in Question 7, above, complete? (Here this could be a report to the sponsor or a scientific publication.) Questions: TRL 4 A. Agreement between technology deliverer and customer: (here the customer is, for instance, the sponsor supporting the research or could be a new program wanting to mature the technology). 1. What are the CTEs? 2. What are the benefits of the new technology? 3. What are the design requirements? These typically include the following. a. Performance/Function (concept of operation, calibration, modes, autonomy, etc.) b. Form/Fit (mass, volume, layout, etc.) c. Interfaces (thermal, mechanical, power, electrical, data, signal/sample input, etc.) d. Operating environments (mechanical, dynamics, thermal, radiation, EMI/EMC, etc.) e. Lifetime 4. What are the relevant environments? 5. What are the analysis requirements? This includes, at minimum, the following: a. Key performance parameters b. Analyze with “first order” equations c. Validation that provides moderate accuracy analysis uncertainty factor and limitations 6. What are the test requirements? Note: Not all design requirements are tested. These include at minimum the following: [SP-20205003605] 23 a. Performance/Function b. Laboratory environment 7. What is the level of integration and test configuration? For TRL 4, at minimum, the component/assembly level is demonstrated by means of a breadboard in the laboratory. 8. What data are used to capture the agreements and results? B. Analysis results: 9. What performance is predicted for the key parameters for the test conditions? Note: these are in place prior to the test. 10. What are the analysis uncertainty factors and limitations? 11. Are the analyses updated based on the test results? 12. What are the life limiting factors? C. Test results: 13. Are the test requirements successfully demonstrated? 14. Are the variances between the test results within the analysis uncertainty? If not, are the variances understood? 15. Were there any unpredicted behaviors? If so, was root cause determined and impact found to be acceptable? D. Data Products: 16. Are the data products, agreed to in Question 8, above, complete? Questions: TRL 5 A. Agreement between technology deliverer and customer: 1. What are the CTEs? 2. What are the benefits of the new technology? 3. What are the design requirements? These typically include the following: a. Performance/Function (concept of operation, calibration, modes, autonomy, etc.) b. Form/Fit (mass, volume, layout, etc.) c. Interfaces (thermal, mechanical, power, electrical, data, signal/sample input, etc.) d. Operating environments (mechanical, dynamics, thermal, radiation, EMI/EMC, etc.) e. Lifetime 4. What are the relevant environments? 5. What are the analysis requirements? This includes the following: a. Key performance parameters and life limiting factors b. Model with “first order” equations c. Validation that provides moderate accuracy analysis uncertainty factor and limitations 6. What are the test requirements? Note: Not all design requirements are tested. These include at minimum the following: a. Performance/Function b. Relevant environments [SP-20205003605] 24 7. What is the level of integration and test configuration? For TRL 5, at minimum, the component/assembly level is demonstrated by means of a brassboard in the relevant environment. 8. What data is used to capture the agreements and results? B. Analysis results: 9. What performance is predicted for the key parameters and life limiting factors for the test conditions? Note: these are put in place prior to the test. 10. What are the analysis uncertainty factors and limitations? 11. Are the analyses updated based on the test results? C. Test results: 12. Are the test requirements successfully demonstrated? 13. Are the variances between the test results within the analysis uncertainty? If not, are the variances understood? 14. Were there any unpredicted behaviors? If so, was root cause determined and impact found to be acceptable? D. Data Products: 15. Are the data products, agreed to in Question 8, above, complete? Questions: TRL 6 A. Agreement between technology deliverer and customer: 1. What are the CTEs? 2. What are the benefits of the new technology? 3. What are the design requirements? Here design requirements should include an agreement on margin above allowed flight parameters. These typically include the following. a. Performance/Function (sensitivity, concept of operation, calibration, modes, autonomy, etc.) b. Form/Fit (mass, volume, layout, etc.) c. Interfaces (thermal, mechanical, power, electrical, data, signal/sample input, etc.) d. Operating environments (mechanical, dynamics, thermal, vacuum, radiation, EMI/EMC, etc.) e. Lifetime 4. What are the relevant environments? 5. What are the analysis requirements? This includes the following. a. Key performance parameters, life limiting factors, lower level of parameters b. Model with “first order” equations c. Validation that provides moderate accuracy analysis uncertainty factor and limitations 6. What are the test requirements? Note: Not all design requirements are tested. These include at minimum the following. a. Performance/Function b. Relevant environments [SP-20205003605] 25 7. What is the level of integration and test configuration? For TRL 6, at minimum, the subsystem level is demonstrated by means of a prototype in the relevant environment. 8. What data are used to capture the agreements and results? B. Analysis results: 9. What performance is predicted for the key parameters and life limiting factors for the test conditions? Note: these are in place prior to the test. 10. What are the analysis uncertainty factors and limitations? 11. Are the analyses updated based on the test results? C. Test results: 12. Are the test requirements successfully demonstrated? 13. Are the variances between the test results within the analysis uncertainty? If not, are the variances understood? 14. Were there any unpredicted behaviors? If so, was root cause determined and impact found to be acceptable? D. Data Products: 15. Are the data products, agreed to in Question 8, above, complete? Questions: TRL 7 A. Agreement between technology deliverer and customer: 1. What are the CTEs? 2. What are the benefits of the new technology? 3. What are the design and test requirements? Here design and test requirements should include an agreement on margin above allowed flight parameters. These typically include the following: a. Performance/Function (concept of operation, calibration, modes, autonomy, etc.) b. Form/Fit (mass, volume, layout, etc.) c. Interfaces (thermal, mechanical, power, electrical, data, signal/sample input, etc.) d. Operating environments (mechanical, dynamics, thermal, radiation, EMI/EMC, etc.) 4. What are the analysis requirements? This includes the following: a. Key performance parameters, life limiting factors, lower level parameters b. Analysis based on physical principles, equations, and statistical methods (use of high-fidelity modeling tools such as finite element analysis structural and thermal codes, detailed optical codes, etc.) c. Validation that provides moderate accuracy analysis uncertainty factor and limitations [SP-20205003605] 26 5. What is the level of integration and test configuration? For TRL 7, at minimum, the subsystem level is demonstrated by means of a prototype demonstrated in space. 6. What data are used to capture the agreements and results? B. Analysis results: 7. What performance is predicted for the key parameters and life limiting factors for the test conditions? Note: these are put in place prior to the test. 8. What are the analysis uncertainty factors and limitations? 9. Are the analyses updated based on the test results? C. Test results: 10. Are the test requirements successfully demonstrated? 11. Are the variances between the test results within the analysis uncertainty? If not, are the variances understood? 12. Were there any unpredicted behaviors? If so, was root cause determined and impact found to be acceptable? D. Data Products: 13. Are the data products, agreed to in Question 6, above, complete? 3.1.5. Roll up of TRL The standard NASA “weakest link” approach is used to determine the TRL of a system. The TRL of a higher level of integration can be no higher than the lowest TRL of its elements. Note, it is possible for a system’s TRL to be lower than that of all its elements. For example, a new architecture that is used to provide new performance might employ all “heritage” parts. To achieve TRL 6, the system would need to be integrated and tested to demonstrate the new performance. However, since the parts are all “heritage,” no environmental verification would be needed provided the heritage parts will operate in an environment identical to their prior qualification and operational environment. 3.2. Technology Readiness Assessment Process 3.2.1 Convening a TRA The requirement for a TRA in NPR 7120.5 is owned by the NASA Office of the Chief Engineer (OCE). This requirement is delegated to the Center Director, per the NASA OCE “Letter of Delegation” located on the OCE tab under the “Other Policy Documents” menu in the NASA On-Line Directives Information System (NODIS). Therefore, each Center is responsible for conducting TRAs for projects for which they are the implementing Center. The TRA can be convened by many means and tailored depending on the needs of the Center as well as the task or project. Examples are below: [SP-20205003605] 27 1. Project/Task may convene a self-assessment within the project/task. 2. Project/Task may convene an independent assessment through the peer review process. 3. Any engineering office may elect to convene an independent TRA review to provide an outside assessment of a technology task or project. 4. The Center may provide institutional certification of TRL through the process documented in this guideline. The convening authority is defined by the Center with involvement by their Technology, Engineering and Office of Safety and Mission Assurance (OSMA) offices. 3.2.2. Relationship of TRAs to Project Life Cycle Technology development associated with a project is identified early in the project life cycle and its maturity level needs to evolve to a confidence level that allows the project to proceed with manageable risk. For NASA operational missions, achieving TRL 6 by PDR has been established as the minimal appropriate maturation level. However, a project with limited resources may be required to prioritize their resources and not achieve TRL 6 based on a risk assessment. This resource evaluation would be made prior to PDR. Other types of projects have established other metrics. For instance, NASA STMD Technology Demonstration Missions have an entry requirement of TRL 5 and a completion requirement of TRL 7, and NASA SMD Technology Demonstration Missions require TRL 6 prior to PDR. Figure 3.2.2-1: TRAs in Operational Mission Life Cycle 3.2.2.1. Operational Missions The recommended guidance is to conduct TRAs during the formulation phase of operational projects, aligned with life cycle reviews (and key decision points), the development of the Technology Maturation Plan (a gate product), and corresponding lifecycle cost analyses. Figure 3.2.2-1 shows the relationship between TRAs and project life cycle reviews. Three TRAs are recommended, with the objective of each evolving Technology Readiness Assessments Technology Readiness Assessments [SP-20205003605] 28 throughout the technology maturation program. The scopes of the three TRAs are given in Table 3.2.2.1-1: Table 3.2.2.1-1: Recommended TRAs for Formulation Process TRA-A TRA-B TRA-C CTEs Initial Final Update if req. Requirements Initial Final Update if req. Maturation Test Plan Initial Final Assess implementation TRL Assessment Assess TRL at project start Assess interim TRL Assess for TRL 6 at PDR 3.2.2.2. Technology Development Projects/Tasks The recommended guidance is to convene TRAs during the concept formulation phase (i.e., proposal submission) and renewal/final review. Figure 3.2.2.2-1 shows the relationship between TRAs and Technology Development Reviews. TRAs are done as part of the proposal/renewal and final reviews. Figure 3.2.2.2-1: TRAs in Technology Development Life Cycle 3.2.3. Technology Readiness Assessment Report The TRA Report documents the description of the process used to conduct the TRA and provides a comprehensive explanation of the assessed TRL for each CTE. The report is delivered to the convening authority. The report should provide citation to and summary descriptions of the objective evidence that support the assessment. This report includes the following information: Life Cycle Phases Concept Formulation Technology Development Concept Completion Technology Program Decision Points Proposal Selection Renewal Approval Technology Transition to Higher TRL Dev (or Flight) Life Cycle Events/Reviews Proposal Submission Renewal Submission Final Report JPL Technology Readiness Assessments TRA-A TRA-B TRA-CTechnology Readiness Assessments [SP-20205003605] 29 1. Synopsis of requirements and agreements between the customer and the technology deliverer. 2. Identification of CTEs. 3. An objective scoring of the level of technology maturity for each CTE by subject matter experts along with the objective evidence for that scoring. 4. Identification of tests and of integration levels required to demonstrate TRL that are captured in the maturation plans for achieving an acceptable maturity roadmap for CTEs prior to critical milestones decision dates. 5. If requested, a risk assessment for each CTE to achieve the required TRL following the standard 5x5 risk assessment matrix. 6. Any additional findings of the assessment review team. [SP-20205003605] 30 4. Risk Assessment Associated with Progression of Maturity Transitioning new technology to flight elements may introduce significant risks. Although TRAs are not designed to determine the risk of a technology, the TRA team may provide insight to inform projects and Programs about the residual risk of a new technology along the development path. TRLs establish the maturity of a new technology at a given time. The degree of difficulty and the risk associated with progressing to higher levels of maturity may vary significantly from one new technology to another. A best practice for the assessment of risk may reside in the use of the risk process outlined in NPR 8000.4, using the standard 5x5 risk matrix developed for flight projects. Best practice for the risk assessment is to use one of the following as established by the TRA convening authority: • Progression from TRL n to TRL n+1 • Progression from current TRL to TRL 6 • Progression from current TRL to TRL 9 Utilize continuous risk management according to NPR 8000.4 throughout the lifecycle. [SP-20205003605] 31 5. Nomenclature/Acronyms AD2 Advanced Degree of Difficulty ARC Ames Research Center CDR Critical Design Review CoFR Certificate of Flight Readiness CTE Critical Technology Element EM-1 Exploration Mission-1 ESD Engineering and Science Directorate at JPL ETD Engineering and Technology Directorate at GSFC GSFC Goddard Space Flight Center HRCR Hardware Review and Certification Record ICPS Interim Cryo-Propulsion Stag IRB Independent Review Board ISS International Space Station JPL Jet Propulsion Laboratory KPP Key Performance Parameter MCR Mission Concept Review MDR Mission Definition Review MUF Model Uncertainty Factor NA Not Applicable NASA National Aeronautics and Space Administration NODIS NASA On-Line Directives Information System NPR NASA Procedural Requirement OCE Office of the Chief Engineer OCT Office of the Chief Technologist OSMA Office of Safety and Mission Assurance OSMS JPL Office of Safety and Mission Success PDR Preliminary Design Review PMSR Program Management Status Review RELL Robotic External Leak Locator SRB Standing Review Board SRR System Requirement Review STMD Space Technology Mission Directorate [SP-20205003605] 32 TRA Technology Readiness Assessment TRL Technology Readiness Level VASMIR Variable Specific Impulse Magnetosphere Rocket VPIST Vacuum Pressure Integrated Suit Test [SP-20205003605] 33 Appendix A. Center TRA Processes Appendix A provides examples of processes three NASA Centers use to conduct TRAs. Each Center can tailor its process for implementing TRAs to the distinct needs of each Center. TRL assessment teams can reference these processes as examples for how to design, modify, and conduct their own TRAs. Note, while the implementation processes can vary, the definitions provided in the best practices guide—such as the TRL definitions—are standard across NASA Centers. [SP-20205003605] 34 Appendix A-1. Goddard Space Flight Center TRA Process A TRL Assessment Team, consisting of the Goddard Space Flight Center (GSFC) Chief Engineer, along with the Engineering and Technology Directorate (ETD) Chief Engineer and ETD Chief Technologist, shall, when requested, perform an independent assessment of TRL. When warranted, the assessment team described above will be augmented with specialized subject matter experts. Proposers/developers desiring an independent TRL assessment from ETD will make a request for the assessment from the GSFC Chief Engineer, the ETD Chief Engineer or the ETD Chief Technologist. TRL assessment requestors shall supply the following info to the TRL Assessment Team: • A block diagram of the system for context of the item being assessed • For the item(s) being assessed: o What is its heritage? o What changes are contemplated with respect to the heritage design? o How do the environments of the heritage application compare with the predicted environment for the item being assessed? o Do either the design changes or the environment changes from the heritage drive the technology in any way? E. A self-assessment (developer/proposer) of the TRL using the criteria in NPR 7123.1, the Heritage Versus Technology Flowchart in the NASA TRL Best Practices Guide, and the questions posed above. The TRL assessment will also consider the risk of the proposed development. The TRL Assessment Team will document their findings, along with rationale, and make this documentation available to the TRL assessment requestors. 1. Component TRL rationale: • Built-to-print flight-proven component, same environment = TRL 9. (Must substantiate with flight history) • Component similar to flight-proven component (no new technology, just new configuration not expected to affect performance; configuration changes are unrelated to the technology) = TRL 7 • Expected Environment similar to flight-environment (difference not expected to have impact on this component) = TRL 7 [SP-20205003605] 35 2. Relevant Environment: Environment relevant to the proposed technology. For example, if the proposed technology has sufficient mass and stiffness that the launch- induced vibro-acoustic environment will not affect it, then the vibro-acoustic environment is not relevant for that technology. 3. System TRL Rationale: F. TRL 9: Same flight-proven components and interfaces, same environment (Must substantiate with flight history.) G. TRL 8: All components have flown, but the identical component configuration (i.e., the mix of components/subsystems) has not flown as a system but: No new technology. Interfacing of mix of components not expected to affect performance and interfacing of components has no relevance to the technology H. TRL 7: Expected Environment of system similar to flight-proven environment (difference not expected to have impact on interfaces or relevance to the technology) I. TRL 6: New technology, new device physics, environment (radiation, launch, thermal, etc.) different in a way that could affect operation, reliability, or mission life, never tested as a system. J. To achieve TRL6 must demonstrate operation as a system in the relevant environment. K. TRL 5: New component technology, new component physics, development process outside demonstrated experience base, environment (radiation, launch, thermal, etc.) different in a way that could affect operation, reliability, or mission life, components never tested in a flight-like environment. [SP-20205003605] 36 Appendix A-2. Ames Research Center TRA Process A-2.1 Introduction The Ames New Opportunities Center (NOC) has implemented a TRL assessment process that will aid Ames PIs is assessing their current instrument/technology TRL. This process was initiated to address the Traffic Management Coordinator Major Weaknesses that Ames consistently receives for instrument TRL and instrument maturation plans when proposing to Science Mission Directorate space mission Announcements of Opportunity. To facilitate this TRL assessment process, the NOC developed checklists for TRL 1 through 6, which are provided herein. Also included is a checklist to assess the TRL maturation plan and the degree of difficulty in advancing to the next TRL. These checklists were formed using information found in various NASA documents, as well guidance from the Department of Defense and European Space Agency TRL documents and calculators. Figure A-2.1. Description of the TRL Assessment Process Step 1: The TRL assessment process begins with the principal investigator (PI) performing a self-assessment of the instrument/technology TRL using the TRL checklists. The PI gathers supporting information as well. (Refer to the TRL checklists for more details on what supporting material is requested for each criterion.) Once compiled, this information is forwarded to the TRL assessment panel for their review. Step 2: The TRL assessment panel then meets with the PI and his/her team to discuss the self-assessment and supporting information. The meeting will focus on functional areas identified on the relevant TRL checklist. Step 3: The TRL assessment process ends with the release of the Independent TRL Assessment. The primary audience of this document is the Center management, the PI, and his/her team, but it will also be shared with relevant proposal review teams. A-2.2 Instructions for Using the TRL checklists 1. Perform a self-assessment of the instrument/technology by using the TRL checklists: review the "Assessment Criteria" of the various TRL checklists to determine the TRL of the instrument/technology. 2. After completing the TRL self-assessment, gather the supporting information identified on the relevant checklist. Note that formal documentation of the supporting information is not necessary. As long as the information is clear to TRL$Self)$ Assessment$by$PI$$ PI$and$TRL$Panel$Meet$ TRL$Panel$Sends$Mgmt.$ the$Independent$TRL$ Assessment$ PI and TRL Assessment Panel Meet [SP-20205003605] 37 reviewers, PowerPoint slides, informal summaries, and/or emails are sufficient to convey the information to the assessment panel. 3. Once the TRL self-assessment is complete and the supporting information has been compiled, email the information to the TRL assessment coordinator, who will then forward the information to the rest of the TRL assessment panel for their review. If the PI would like an assessment of the instrument/technology maturation plan, compile the information requested on the last checklist and submit it along with the other information for the TRL assessment. 4. This checklist assesses the TRL maturation plan and is primarily intended for use on instruments/technology in the range of TRL 3 to 6. If the PI would like an assessment of the maturation plan, compile the information requested on this checklist, and submit it along with the other information for the TRL assessment. Functional Area Maturation Plan Assessment Criteria Supporting Information Technical Plans 1. Current TRL and exit TRL identified. 2. Fidelity of hardware to be built and tested. Differences from flight unit documented. 3. Plans for analytical predictions of hardware performance identified. 4. Plans for modeling and simulations identified (if applicable). 5. Test plan complete, including test set-up, required equipment/facilities, test environment, samples tested, and total test hours required. • Current/exit TRL • Hardware fidelity, including form, fit, function, & scale • Planned key trade studies • Planned analytical predictions • Planned modeling and simulations (if applicable) • Test plan • Differences between this hardware & flight unit • Differences between operating environment & test environment Cost 6. Cost estimate complete. 7. Potential funding source(s) identified. • Budget and BOEs • Phasing plan (personnel required) • Expected funding source Schedule 8. Maturation plan timeframe and duration identified. • Development plan schedule • Major milestones • Long-lead items Risks & Descope Plan 9. Risks to completing the plan identified. 10. Options, alternatives, and descopes to the development plan identified. • Risk list • Options, alternatives, & descopes [SP-20205003605] 38 A-2.3 TRL Assessment Process Template Instrument: Instrument Name PI: PI Name Date: Report Date Mission/Use: Expected mission or use Instrument Type: High-level instrument description TRL Assessment Panel Members: Panel members Material Reviewed by the Panel: List the supporting information provided by the PI TRL Assessment: Identify the panel's recommended TRL # Strengths • List all the strengths identified by the panel Weaknesses • List all the weaknesses identified by the panel Recommendations and Comments • List any recommendations or comments the panel has for the PI and/or Center management. [SP-20205003605] 39 Appendix A-3. Jet Propulsion Laboratory TRA Process A-3.1. Convening a TRA The requirement for a TRA is delegated to the Center Director, per the NASA OCE "Letter of Delegation.” Therefore, JPL is responsible for conducting TRAs for projects for which JPL is the implementing center. The TRA can be convened by many means and tailored depending on the needs of the task or project. Examples are below. 1. Project/Task may convene a self-assessment within the project/task. 2. Project/Task may convene an independent assessment through the JPL peer review process. 3. The JPL Program Directorate may elect to convene an independent TRA review to provide an outside assessment. 4. JPL may provide institutional certification of TRL through the process documented in this document. The convening authority is then the Engineering and Science Directorate (ESD) with the Office of the Chief Technologist (OCT) for TRL 1-5, and ESD with the OCE and Office of Safety and Mission Success (OSMS) for TRL 6-9. A-3.2. Program or Institutionally Convened Independent TRA Review Board If the program or JPL institution convenes an independent TRA review board, then the chair is a JPL person ideally on the project Standing Review Board (SRB) or Independent Review Board (IRB) who then accepts dual roles. The chair provides a TRA report for the conveners and the JPL OCE/OCT/ESD/OSMS and provides input to the SRB or IRB during technology maturation. The independent TRA review board serves in an advisory role that provides recommendations to the convening authority, but does not impose requirements on or make decisions for the project. The convening authority selects the TRA Review Board Chair. The TRA Review Board chair selects the independent TRA review board members, who are approved by the convening authority. Its composition should include both broad area expertise familiar with the project and subject matter experts familiar with the new technologies. The independent TRA review board members are selected from outside the project. Typically, these members are from the Program Office, ESD Chief Technologists, and Chief Engineers or from experts either inside or outside of JPL. The TRA review board chair and/or board members may be invited to the project life cycle reviews during formulation and may interact with the project between TRA-A and TRA-B to provide input to finalize the Technology Maturation Plan. The independent TRA review board remains intact through project formulation, completing its activities after the Project PDR. This provides continuity and familiarity with the projects purpose, history, technical approach, challenges, and risks.