EVOLUTION OF SHUI - rLE AVIONICS REDUNDANCY MANAGEMENT / FAULT TOLERANCE Jack
چکیده
The challenge of providing redundancy management (RM) and fault tolerance to meet the Shuttle Program requirements of fail operational/fail safe for the avionics systems was complicated by the critical program constraints of weight, cost, and schedule, This paper addresses the basic and sometimes false effectivity of less than pure RM designs. Evolution of the multiple input selection filter (the heart of the RM function) is discussed with emphasis on the subtle interactions of the flight control system that were found to be potentially catastrophic. Several other general RM development problems are discussed, with particular emphasis on the inertial measurement unit RM, indicative of the complexity of managing that three-string system and its critical interfaces with the guidance and control systems. PROGRAM REQUIREMENTS AFFECTING FAULT TOLERANCE/RM Space Shuttle Program requirements dictate fault tolerance In all systems other than primary structure, thermal protection, and pressure vessels. In the case of avionics systems, those requirements are specified to be two-fault tolerant, or fail operational/fall safe (FO/FS) (ref. 1) as commonly referred to in Shuttle terms. Considering the design life goals of 100 missions and 10 years or more operational span, this FO/FS requirement not only appears reasonable but almost minimally mandatory. Given a free hand in hardware provisioning, meeting the FO/FS requirement may have been obtainable. As would be expected in a program heavily pressed with cost and weight constraints, the avionics designers' hands were not so free. Providing FO/FS fault tolerance with no holes or subtle escapes would most easily be accomplished by providing independent five-string operation with independent data inputs to each of the five strings and command functions requiring three out of five votes before execution. This approach would obviously be costly in weight and cost of wiring and avionics components but does offer an FO/FS design approach that could be vold of in-flight failure effects concerns for two failures, it would also eliminate the historically complex hardware/software requirements for fault detection and isolation since the first two failures would be transparent. This five-string approach Is indeed being implemented In the flight-critlcal interfaces of the Centaur payloads with the Orbiter. The approach was never seriously considered for Shuttle/Orbiter design because of the obvious systems weight and cost Impacts, yet the FO/FS requirement was maintained. The first step down from the five-string design to meet FO/FS is to provide four-strlng operation with fault detection and isolation of at least the first fault. Various schemes can be implemented at the four-string level to choose proper data or perform the proper command function, but the susceptibility to the second failure requires a fault detection and isolation scheme to eliminate the first failure before occurrence of the second failure In order to be FO/FS. Two simultaneous or near-simultaneous (within the timespan required to detect and isolate the first failure) failures will defeat this approach; however, Shuttle Program management has granted detection and reconfiguration tlme in Its FO/FS requirement provided the detection Is reasonably assured of working. In this scheme, the second failure Is then tolerated by selecting the middle valued data of the remaining three inputs and voting two of three for command functions or providing enough muscle in the two good command channels to override the second failure channel. This four-level approach was selected for a major portion of the Shuttle avionics and has fared well with the possible exception of null failures in sensors normally operating about their null, thereby hindering the fault detection of either the first or the second null failure. The next step down in redundancy while maintaining FO/FS fault tolerance is to provide threestring operation in conjunction with fault detection and isolation of both the first and second failures in the system. Detecting and isolating the first failure In a three-strlng system is not overly complex and detecting the second failure through two-level comparison of inputs or outputs Is straightforward. The difficulty in this approach comes in isolating the second failure. Disagreement between only two inputs requires a decisionmaking vote that is often costly and still not 100 percent certain, suchasbuilt-in test equipment (BITE),self-tests, or reasonableness te ts. In thecases whereisolation is still lacking,a crew"guesstimate" andmanual reconfiguration maybeusedto supplement the auto RM. This approach can also be costly and leaves room for errors. This second fault isolation problem was a primary factor in the basic four-string avionics design in the Shuttle with the Orbiter inertial measurement units (IMU's) being the most notable exception. The basis for this exception, as with the other three-strlng systems, included weight and cost, reliability background and history of similar hardware, and projected BITE capabilities to cover second failure isolation. Treatment of the second failure detection and isolation of the IMU's has involved such extensive efforts in analysis, verification, software changes, and flight procedures development that the possibility of adding a fourth IMU to the Orbiter is still under consideration. Indeed, the complexities of timely detection and isolation of two-level IMU faults and the capabilities and limitations of the IMU BITE design are sufficient to warrant treatment as an individual paper, accounting for the large portion of this paper being dedicated to IMU RM. A considerable amount of the IMU RM design change activity deals with fine tuning of the data to minimize errors passed to the guidance and control systems; however, the final fault tolerance assessment accounting for the two-level isolation problems results in no capability to isolate approximately 0.7 percent of the second faults and an even more disconcerting 0.15 percent of the second faults which can result in selection of the failed IMU (ref. 2). Any further reduction in redundancy level below three strings obviously fails to meet the avionics FO/FS requirement unless, of course, the function itself is not required and is then by definition FS after the second failure. The net result is that Shuttle avionics basically evolved as a threeand four-strlng design attempting to provide fault detection and isolation capability to make these redundancy levels equal to the five-string design required to be purely FO/FS. The success of this design is not easily measured; however, there are several key data points to recount when considering Shuttle redundancy design versus a blanket FO/FS requirement. First, the Orbiter avionics alone has officially documented some 255 critical-items lists (CIL) exceptions to the FO/FS requirement. Secondly, in striving to meet the blanket FO/FS requirement, the less than pure approach has resulted in an analysis and verification program of staggering proportions. As former astronaut and Orbital Flight Test Manager Donald K. Slayton stated (ref. 3), though possibly too late to influence the Shuttle Program direction, there is an unpredictable but "high cost of worrying improbable possibilities." Because of the second fault tolerance requirement and the fault detection and isolation deficiencies (delays in isolation and/or various degrees of escapes), every combination of failures, however improbable, must be analyzed and verified as acceptable. This is then magnified by reconfiguration actions planned to optimize the postfailure system configuration, each of these obviously requiring verification. In this process, as is its objective, escapes from the FO/FS requirement are sometimes discovered, leading to software, hardware, and/or procedural changes which, of course, add to the verification work. Quantification of such changes is nearly impossible but some understanding of the problem can be evidenced by the ever-present hardware and software change boards, the approximately 700 pages of crew malfunction procedures, the more than 1000 pages of offnominal crew procedures, the libraries full of off-nominal verification procedures and test reports, and the more than 250 software change requests which have currently been approved to the RM Flight System Software Requirements (FSSR) alone (ref. 4). Mr. Slayton's approach to the problem was proposed basically as drawing a line in the sand defining some failure criticality not based on a blanket FO/FS requirement but rather based on the probability of that failure (or combination of failures) to occur. If the failure probability is determlned to be lower than that llne, the design would not have to accommodate changes to meet the FO/FS requirement. The obvious problems of choosing the acceptable failure probability and the mechanization of determining the probability of potential failures are admittedly nontrivial; however, Mr. Slayton cites the success of the Saturn 5 program as a working example of this approach. An alternate approach that has essentially been described earlier is to provide a five-strlng operation with data and command selection, based on the middle value of the five or voted to require three of the five data or command statuses to produce a functional response. The aforementioned weight and cost penalties of such a system are understandably undesirable and easily considered unjustified in the preliminary stages of system design. Considering the historically costly change traffic and verification activities that go with the attempted development of less than pure redundancy matching requirements, the initial acceptance of the additional string weight and cost may have been an effective overall decision for the Shuttle/Orbiter Program. IMPACTS OF SUBSYSTEM PERFORMANCE CHARACTERISTICS ON RM The Shuttle RM involves both hardware and software functional implementations, the extent of each determined by the design and response characteristics of the individual subsystem element. As with any development program on the level of sophistication and gross technical expansion of functional capabilities such as in the Shuttle Program, the potentials for hardware and software design deficiencies are not few. Adding the scheduling demands typically forced on NASA programs compllcates the situation by requiring parallel development of subsystem hardware and the associated applications and RM software. This parallel RM design does not then allow for an individual llne replaceable unit (LRU) performance understanding beyond that established as design requirements in the procurement specifications. Additionally, the benefits of failure modes and effects analyses on the LRU's cannot be utilized in initial designs, since these analyses are not completed sufficiently until final design and/or deslgn verification testing has been completed. Finally, the performance of the LRU's within the system as a whole cannot be adequately anticipated until integrated systems slmulatlons and/or verification programs are underway involving flight hardware/software designs and sophisticated models of environments, aerodynamic effects, and vehicle dynamic models. Development of the Ridschemes under these handicaps can and did result in many initial assumptions that turned out to be invalid or at least not sufficient to provide totally acceptable RM. A primary example of this programmatic dilemma occurred in the Orbiter flight control system (FCS) interface with the redundant FCS sensor systems. Providing the FCS with the best available rate and acceleration data from a quad redundant sensor set at first glance appeared trivial. The program had provided the fourth sensor set after only three had been initially proposed because it was recognized that isolation of the second failure in dynamic periods of flight could not be guaranteed within FCS control limits. The formulation and use of a software-derived fourth input was too complex and laced with its own shortcomings. The selection of the "best" data input from this set was considered to be a simple extension of the three-sensor method; i.e., provide a software selection filter (SF) which dealt wlth the first three inputs by simply selecting the middle valued input for use by the FCS, substituting the fourth sensor input only after one of the original three had been determined to be failed (fig. 1). This SF design was baselined and implemented In the software RM along with a fault detection, isolation, and reconflguration (FDIR) scheme that used a unit-to-unlt comparison test to determine whether a unit had failed. On the surface, this approach was simple and foolproof; in final application, it was not so simple and indeed inadequate. Two key conditions co_bined to defeat this original approach: (I) to eliminate or limit false alarm exposure, the magnitude of the differences between units had to be significantly large enough to account for unfailed unit normal deviations (30), system noise, and transients; and (2) the FCS control laws and vehicle responses aimed toward and generally achieved stable flight conditions within the limits of times and/or magnitudes established to prevent the false alarms. The interaction of these two factors resulted in a general inability to detect and isolate a null failed sensor (fig. 2).