Modular Neural Networks for Medical Prognosis: Quantifying the Benefits of Combining Neural Networks for Survival Prediction
چکیده
This paper describes a medical application of modular neural networks for temporal pattern recognition. In order to increase the reliability of prognostic indices for patients living with the Acquired Immunodeficiency Syndrome (AIDS), survival prediction was performed in a system composed of modular neural networks that classified cases according to death in a certain year of follow-up. The output of each neural network module corresponded to the probability of survival in a given year. Inputs were the values of demographic, clinical, and laboratory variables. The results of the modules were combined to produce survival curves for individuals. The neural networks were trained by backprogation and the results were evaluated in test sets of previously unseen cases. We showed that, for certain combinations of neural network modules, the performance of the prognostic index, measured by the area under the receiver operating characteristic (ROC) curve, was significantly improved ( p <0.05). We also used calibration measurements to quantify the benefits of combining neural network modules, and show why, when, and how neural networks should be combined for building prognostic models. Introduction The most common applications of neural networks in clinical medicine have been for diagnosis of diseases (Reggia, 1994). While diagnostic tasks usually require classification of cases for one time point, prognostic tasks may require the assessment of a patient’s status over time. A few reports on the use of neural networks in prognostic tasks have been published. These systems determine prognosis after cardiopulmonary resuscitation (Ebell, 1993), strategies for weaning from respiratory support (Ashutosh et al., 1992), tumor stage in oncology (Burke, 1994), graft outcome after liver transplantation (Doyle et al., 1994), prognosis in trauma (McGonigal et al., 1993), and prognosis for patients with breast cancer (Ravdin & Clark, 1992). Most of these neural network models assessed prognoses at a single point in time (e.g., death in five years), and did not produce a prognostic survival curve. Outcome prediction is not an easy task in medicine, especially if independent assessments for different time points have to be combined in a meaningful survival curve. The decomposition of a classification task into small subcomponents facilitates learning in neural network classification systems. This approach can be used in prognostic survival analysis, which can be viewed as a type of classification problem. In each neural network module for survival analysis, probabilities of being alive are produced for an individual at a given point in time (e.g., 50 percent probability of being alive in one year). The resulting probabilities can be plotted and extrapolated to produce continuous prognostic survival curves for individuals. However, since each module predicts survival in a given point in time in isolation, nonmonotonic survival curves can be produced, as shown in Figure 1. In order to decrease the frequency of nonmonotonic curves (and also the frequency of nonmonotonic intervals within a curve), we can use the results of certain modules to create a combined neural network, as illustrated in Figure 2. In this example, predictions for years 1 and 3 are used as inputs to a neural network that predicts survival in year 2, and a monotonic survival curve is produced. The dependencies between predictions of each module are explicitly modeled in a combined neural network. Limited resources preclude the utilization of all possible module combinations. In this paper, we show how certain combinations provide higher predictive performance increments in terms of calibration and resolution. We used the domain of HIV infection to illustrate the use of our modular neural networks. Material and Methods The ATHOS database is a longitudinal, primary data set of HIV+ and at-risk subjects, collected from 10 clinics in California (3 private practices in the San Francisco Bay Area, 2 private practices in Los Angeles, and 5 community clinics associated with the Owen Clinic at University of California–San Diego) (Fries, 1992). The ATHOS database was built to provide a national HIV data resource that permits the systematic study of (1) disease costs and financing, (2) drug effectiveness, toxicity, and cost, (3) delivery systems and practice variations, (4) health status and quality of life, and (5) disease transitions and modeling. The data collection began in 1989 at Stanford University and involved abstraction of existing charts, as well as prospective collection of clinical and laboratory information. A modular system of neural networks was used to predict survival for people living with AIDS. This system was composed of several feed-forward networks trained by backpropagation. This architecture was chosen because we were getting poor accuracy when isolated neural networks were used. The subset of the ATHOS data set used in this study consisted of 914 patients. Missing values were imputed as their means (continuous values), or mode (discrete values). Models were constructed using all patients. The bootstrap method (100 replicates) was used for evaluation (Efron, 1979). The number of patients and the distribution of AIDS deaths according to year of follow-up are show in Table I. The balance of cases in each set is defined as balance = min(AIDS deaths/Total, AIDS survivals/Total) A balance of 0.5 means that the proportion of cases is 1:1, and is the one that best facilitates classification by statistical and neural network models, as will be shown later.