In this paper we first revisit a classical problem of computing variational splines. We propose to compute local variational splines in the sense that they are interpolatory splines which minimize the energy norm over a subinterval. We shall show that the error between local and global variational spline interpolants decays exponentially over a fixed subinterval as the support of the local vari...

Mixture models for random graphs have a complex dependency structure and a likelihood which is not computable even for moderate size networks. Variational and variational Bayes techniques are useful approaches for statistical inference of such complex models but their theoretical properties are not well known. We give a result about the consistency of variational estimates of the parameters of ...

Generally speaking, there exist two basic ways to describe a physical problem [1]: (1) by differential equations (DE) with boundary or initial conditions; (2) by variational principles (VP). The VP model has many advantages over its DE partner: simple and compact in form while comprehensive in content, encompassing implicitly almost all information characterizing the problem under consideration...

We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout [10]. Stochastic gradient variational Bayes (SGVB) [9] is a general framework for estimating the evidence lower bound (ELBO) in Variational Bayes. In this work, we extend the SGVB estimator with using Alpha-Divergences, which are alternative to divergences to...

Variational inference is an umbrella term for algorithms which cast Bayesian inference as optimization. Classically, variational inference uses the Kullback-Leibler divergence to define the optimization. Though this divergence has been widely used, the resultant posterior approximation can suffer from undesirable statistical properties. To address this, we reexamine variational inference from i...

Bayesian learning is often accomplished with approximation schemes because it requires intractable computation of the posterior distributions. In this paper, focusing on the approximation scheme, variational Bayes method, we investigate the relationship between the asymptotic behavior of variational stochastic complexity or free energy, which is the objective function to be minimized by variati...

For variational inequalities, various merit functions, such as the gap function, the regularized gap function, the D-gap function and so on, have been proposed. These functions lead to equivalent optimization formulations and are used to optimization-based methods for solving variational inequalities. In this paper, we extend the regularized gap function and the D-gap functions for a quasi-vari...

In this paper, we introduce a new class of variational inclusions involving three operator. Using the resolvent operator technique, we establish the equivalence between the general variational inclusions and the resolvent equations. We use this alternative equivalent formulation to suggest and analyze some iterative methods for solving the general variational inclusions. We also consider the cr...

Recent progress in variational inference has paid much attention to the flexibility of variational posteriors. Work has been done to use implicit distributions, i.e., distributions without tractable likelihoods as the variational posterior. However, existing methods on implicit posteriors still face challenges of noisy estimation and can hardly scale to high-dimensional latent variable models. ...