Bayesian Normalized Gaussian Network and Hierarchical Model Selection Method

This paper presents a variational Bayes (VB) method for normalized Gaussian network, which is a mixture model of local experts. Based on the Bayesian framework, we introduce a meta-learning mechanism to optimize the prior distribution and the model structure. In order to search for the optimal model structure efficiently, we also develop a hierarchical model selection method. The performance of our method is evaluated by using function approximation problems and an system identification problem of a nonlinear dynamical system. Experimental results show that our Bayesian framework results in the reduction of generalization error and achieves better function approximation than existing methods within the finite mixtures of experts family when the number of training data is fairly small.