The Role and Quality of Software Safety in the NASA Constellation Program

For NASA quality assurance managers, obtaining an accurate, program-wide picture of software safety risk is difficult across the multiple, independently-developing systems in the NASA Constellation program. In this study, we create metrics that leverage one source of safety information, hazard analysis, to provide NASA quality assurance managers with information regarding the ongoing state of software safety. The goal of this research was two-fold: 1) to quantify the importance of software with respect to system safety; and 2) to quantify the level of risk presented by software in the hazard analysis. We examined 154 hazard reports created during the preliminary design of three major flight hardware systems in the Constellation program. To quantify the importance of software, we collected metrics based on the number of software-related causes and controls of hazardous conditions. To quantify the level of risk presented by software, and we created and applied a metric scheme to measure the specificity of software causes descriptions. We found that 49-70% of hazardous conditions in the three systems could be caused by software or software was involved in the prevention of the hazard. We also found that 12-17% of the 2013 hazard causes involved software, and that 23-29% of all causes had a software control. Furthermore, 10-12% of all controls were software-based. We applied our metrics for measuring the specificity of software causes and found that the results varied greatly between projects. The application of our software specificity metrics identified risks in the hazard reporting process. Software causes are not consistently scoped, and the presence of software in a cause or control is not always clear. Furthermore, a number of traceability risks were present in the hazard reports that could impede verification of software and system safety.