ویراساینس.
سند علمی

A compressed sensing approach for efficient ensemble learning

Lin Li, Rustam Stolkin, Licheng Jiao, Fang Liu, Shuang Wang

چکیده

This paper presents a method for improved ensemble learning, by treating the optimization of an ensemble of classifiers as a compressed sensing problem. Ensemble learning methods improve the performance of a learned predictor by integrating a weighted combination of multiple predictive models. Ideally, the number of models needed in the ensemble should be minimized, while optimizing the weights associated with each included model. We solve this problem by treating it as an example of the compressed sensing problem, in which a sparse solution must be reconstructed from an underdetermined linear system. Compressed sensing techniques are then employed to find an ensemble which is both small and effective. An additional contribution of this paper, is to present a new performance evaluation method (a new pairwise diversity measurement) called the roulette-wheel kappa-error. This method takes into account the different weightings of the classifiers, and also reduces the total number of pairs of classifiers needed in the kappa-error diagram, by selecting pairs through a roulette-wheel selection method according to the weightings of the classifiers. This approach can greatly improve the clarity and informativeness of the kappa-error diagram, especially when the number of classifiers in the ensemble is large. We use 25 different public data sets to evaluate and compare the performance of compressed sensing ensembles using four different sparse reconstruction algorithms, combined with two different classifier learning algorithms and two different training data manipulation techniques. We also give the comparison experiments of our method against another five state-of-theart pruning methods. These experiments show that our method produces comparable or better accuracy, while being significantly faster than the compared methods. & 2014 Elsevier Ltd. All rights reserved.

اسناد مشابه

  • 1-Bit Compressive Sensing for Efficient Federated Learning Over the Air

    مقاله‌ی دسترسی آزاد۲۰۲۳IEEE Transactions on Wireless Communications['Xin Fan', 'Yue Wang', 'Yan Huo', 'Zhi Tian', '']

    For distributed learning among collaborative users, this paper develops and analyzes a communication-efficient scheme for federated (FL) over the air, which incorporates 1-bit compressive sensing (CS) into analog-aggregation transmissions. To facilitate design parameter optimizat

  • A Compressive Sensing Approach for Federated Learning Over Massive MIMO Communication Systems

    مقاله‌ی دسترسی آزاد۲۰۲۱IEEE Transactions on Wireless Communications['Yo-Seb Jeon', 'Mohammad Sadegh Amiri', 'Jun Li', 'H. Vincent Poor', '']

    Federated learning is a privacy-preserving approach to train global model at central server by collaborating with wireless devices, each its own local training data set. In this paper, we present compressive sensing for federated over massive multiple-input multiple-output commun

  • SWITCH: A Novel Approach to Ensemble Learning for Heterogeneous Data

    سند علمی۲۰۰۴ECMLRong Jin, Huan Liu

    The standard framework of machine learning problems assumes that the available data is independent and identically distributed (i.i.d.). However, in some applications such as image classification, the training data are often collected from multiple sources and heterogeneous. Ense

  • Image Compressive Sensing Recovery Using Adaptively Learned Sparsifying Basis via L0 Minimization

    سند علمی۲۰۱۴Signal ProcessingJian Zhang, Chen Zhao, Debin Zhao, Wen Gao

    Frommany fewer acquired measurements than suggested by the Nyquist sampling theory, compressive sensing (CS) theory demonstrates that, a signal can be reconstructed with high probability when it exhibits sparsity in some domain. Most of the conventional CS recovery approaches, ho

  • Improving Data Integrity with Randomness – A Compressive Sensing Approach

    سند علمی۲۰۰۹Zainul Charbiwala, Sadaf Zahedi, Younghun Kim, Supriyo Chakraborty

    Data loss in wireless sensor systems is inevitable, either due to exogenous (such as transmission medium impediments) or endogenous (such as faulty sensors) causes. While there have been many attempts at coping with this issue, recent developments in the area of Compressive Sensi

  • An Efficient Ensemble Learning Method for Gene Microarray Classification

    سند علمی۲۰۱۳BioMed research internationalAlireza Osareh, Bita Shadgar

    The gene microarray analysis and classification have demonstrated an effective way for the effective diagnosis of diseases and cancers. However, it has been also revealed that the basic classification techniques have intrinsic drawbacks in achieving accurate gene classification a