Sparse Regression Modelling Using an Incremental Weighted Optimization Method Based on Boosting with Correlation Criterion

ABSTRACT A novel technique is presented to construct sparse Gaussian regression models. Unlike most kernel regression modelling methods, which restrict kernel means to the training input data and use a fixed common variance for all the regressors, the proposed technique can tune the mean vector and diagonal covariance matrix of individual Gaussian regressor to best fit the training data based on the correlation between the regressor and the training data. An efficient repeated weighted optimization method is developed based on boosting with the correlation criterion to append regressors one by one in incremental regression modelling. Experimental results obtained using this construction technique demonstrate that it offers a viable alternative to the existing state-ofart kernel modelling methods for constructing parsimonious regression models. Index Terms — Regression, construction algorithm, correlation, mean square error, boosting