The evidence framework applied to sparse kernel logistic regression

In this paper we present a simple hierarchical Bayesian treatment of the sparse kernel logistic regression (KLR) model based on the evidence framework introduced by MacKay. The principal innovation lies in the re-parameterisation of the model such that the usual spherical Gaussian prior over the parameters in the kernel induced feature space also corresponds to a spherical Gaussian prior over the transformed parameters, permitting the straight-froward derivation of an efficient update formula for the regularisation parameter. The Bayesian framework also allows the selection of good values for kernel parameters through maximisation of the marginal likelihood, or evidence, for the model. Results obtained on a variety of benchmark datasets are provided indicating that the Bayesian kernel logistic regression model is competitive with kernel logistic regression models, where the hyper-parameters are selected via cross-validation and with the support vector machine and relevance vector machine.