While the general theory of recursive Bayesian estimation of dynamic models is well developed, its practical implementation is restricted to a narrow class of models, typically models with linear dynamics and Gaussian stochastics. The theoretically optimal solution is infeasible for non-linear and/or non-Gaussian models due to its excessive demands on computational memory and time. Parameter es...

Under the framework of the Kullback-Leibler (KL) distance, we show that a particular case of Gaussian probability function for feedforward neural networks (NNs) reduces into the first-order Tikhonov regularizer. The smooth parameter in kernel density estimation plays the role of regularization parameter. Under some approximations, an estimation formula is derived for estimating regularization p...

We address the stochastic characteristics of a recently proposed greedy routing scheme. The behavior of individual hop advancements is examined, and asymptotic expressions for the hop length moments are obtained. The change of the hop distribution as the sink distance is varied is quantified with a Kullback–Leibler analysis. We discuss the effects of the assumptions made, the inherent dependenc...

In this paper I develop two new multivariate dependence measures to aid in the analysis of neural population coding. The first, which is based on the Kullback-Leibler distance, results in a single value that indicates the general level of dependence among several random variables. The second, which is based on an orthonormal series expansion of the joint density function, provides more detail a...

In the context of Bayesian Probability Theory, we discuss a model for estimating the plasma ion effective charge Zeff , integrating data from both bremsstrahlung spectroscopy and individual impurity concentrations obtained via Charge Exchange Spectroscopy (CXS). The validity of the model, taking into account statistical as well as systematic uncertainties, is shown via the Deviance Information ...

Adaptive modulation is a method to increase the data capacity, throughput and efficiency of time division duplexed wireless communication systems. In adaptive modulation, the transmitter continually monitors the dynamic channel and adjusts the transmission parameters such as modulation order, coding rate etc. accordingly. Blind modulation detection schemes play an important role in adaptive sys...

In this paper we formulate in general terms an approach to prove strong consistency of the Empirical Risk Minimisation inductive principle applied to the prototype or distance based clustering. This approach was motivated by the Divisive Information-Theoretic Feature Clustering model in probabilistic space with Kullback-Leibler divergence which may be regarded as a special case within the Clust...

The Metropolis-Hastings (MH) algorithm is a way to sample a provided target distribution π(x). It works by repeatedly sampling a separate proposal distribution T (x, x) to generate a random walk {x(t)}. We consider a modification of the MH algorithm in which T is dynamically updated during the walk. The update at time t uses the {x(t < t)} to estimate the product distribution that has the least...

In this paper we consider weighted nonnegative matrix factorizations and we show that the popular algorithms of Lee and Seung can incorporate such a weighting. We then prove that for appropriately chosen weighting matrices, the weighted Euclidean distance function and the weighted generalized Kullback-Leibler divergence function are essentially identical. We finally show that the weighting can ...

We propose a compact descriptor of video contents based on modeling the temporal behavior of image features using coupled Markov chains. The framework allows us to combine multiple features within the same model, including the representation of the dependencies and relationships between them. The Kullback-Leibler divergence stands out as the base of a perceptually significant distance measure f...