Quantitative Comparison of Linear and Non-linear Dimensionality Reduction Techniques for Solar Image Archives
چکیده
This work investigates the applicability of several dimensionality reduction techniques for large scale solar data analysis. Using the first solar domain-specific benchmark dataset that contains images of multiple types of phenomena, we investigate linear and nonlinear dimensionality reduction methods in order to reduce our storage costs and maintain an accurate representation of our data in a new vector space. We present a comparative analysis between several dimensionality reduction methods and different numbers of target dimensions by utilizing different classifiers in order to determine the percentage of dimensionality reduction that can be achieved on solar data with said methods, and to discover the method that is the most effective for solar images. Introduction In this work, we present our dimensionality reduction analysis aimed towards the ambitious goal of building a largescale Content Based Image Retrieval (CBIR) system for the Solar Dynamics Observatory (SDO) mission [1]. Our motivation for this work comes from the fact that with the large amounts of data that the SDO mission started transmitting, hand labeling (commonly used by solar physicist in the last decades) of these images is simply impossible. There have been several successful CBIR systems for medical images [2] as well as in other domains [3]; none of them, however, have dealt with the volume of data that the SDO mission generates. This NASA mission, only with its Atmospheric Imaging Assembly (AIA), generates eight 4096 pixels x 4096 pixels images every 10 seconds. This leads to a data transmission rate of approximately 700 gigabytes per day only from the AIA component (the entire mission is expected to be sending about 1.5 terabytes of data per day, for a minimum of 5 years). With such a massive pipeline choosing redundant dimensions on our data will lead to unnecessary data storage, and high search and retrieval costs in our repository. Based on these complications, one of the main goals of this work is to determine the percentage of dimensionality reduction we can achieve using the best methods while maintaining a Copyright © 2012, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. high quality parameter-based representation of the solar images. Dimensionality reduction methods have been shown to produce accurate representations of high dimensional data in a lower dimensional space with very domain specific results in other image retrieval application domains [4-8]. In this work we investigate four linear and four non-linear dimensionality reduction methods with eight different numbers of target dimensions as parameters for each, in order to present a comparative analysis. The novelty of our work is to determine which dimensionality reduction methods produce the best and most consistent results and with which classifiers, on our specific image parameters selected for solar data [17]. Due to domain-specific results, reported by multiple-researchers working on dimensionality reduction in the past [4-8], we believe our results will be of special interest to researchers from the field of medical image analysis, as these images seem to be the closest to our dataset [21. We also identify some interesting combinations of dimensionality reduction methods and classifiers that behave differently across the presented datasets. Our research problem in Solar physics is of great practical relevance for Earth's climate since solar flares endanger the lives of passengers on commercial airline routes going over the poles, interrupt radio communications in bands the military uses, can (and have) knocked down power grids. The systematic feature recognition and the study of the metadata, is a key component of the ultimate prediction of solar activity (space weather). The rest of the paper is organized in the following way: A background overview is presented in the following section. After that we present an overview of the steps and experiments we performed together with our observations. The last includes our conclusions and the future work we propose to complete in order to prepare all parts of a solar CBIR system for integration.