Bayesian Regularization via Graph Laplacian

Regularization plays a critical role in modern statistical research, especially in high dimensional variable selection problems. Existing Bayesian methods usually assume independence between variables a priori. In this article, we propose a novel Bayesian approach, which explicitly models the dependence structure through a graph Laplacian matrix. We also generalize the graph Laplacian to allow both positively and negatively correlated variables. A prior distribution for the graph Laplacian is then proposed, which allows conjugacy and thereby greatly simplifies the computation. We show that the proposed Bayesian model leads to proper posterior distribution. We also connect our method with some existing regularization methods, such as Elastic Net, Lasso, OSCAR and Ridge regression. For posterior computation, we develop an efficient MCMC method based on parameter augmentation. Finally, we demonstrate the method through simulation studies and real data analysis.