Information-theoretic measures such as the entropy, cross-entropy and the KullbackLeibler divergence between two mixture models is a core primitive in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably does not admit a closed-form formula, it is in practice either estimated using costly Monte-Carlo stochastic integration, approximated, or bounded using vari...

We introduce a stochastic approximation method for the solution of an ergodic Kullback-Leibler control problem. A Kullback-Leibler control problem is a Markov decision process on a finite state space in which the control cost is proportional to a Kullback-Leibler divergence of the controlled transition probabilities with respect to the uncontrolled transition probabilities. The algorithm discus...

In this paper, we provide a detailed analysis of the global convergence properties of an extensively studied and extremely effective fixed-point algorithm for the Kullback–Leibler approximation of spectral densities, proposed by Pavon and Ferrante in [1]. Our main result states that the algorithm globally converges to one of its fixed points. Index Terms Approximation of spectral densities, spe...

Reply to " Comments on Kullback-Leibler and renormal-ized entropies: Applications to electroencephalograms of epilepsy patients ". Abstract Kopitzki et al (preceeding comment) claim that the relationship between Renormalized and Kullback-Leibler entropies has already been given in their previous papers. Moreover, they argue that the first can give more useful information for e.g. localizing the...

This paper demonstrates the performance of two possible CAT selection strategies for cognitive diagnosis. One is based on Shannon entropy and the other is based on Kullback-Leibler information. The performances of these two test construction methods are compared with random item selection. The cognitive diagnosis model used in this study is a simplified version of the Fusion model. Item banks a...

Several methods that use the notion of viewpoint quality have been recently introduced in different areas of computer graphics, such as scene understanding, exploration of virtual worlds, radiosity and global illumination, image-based rendering and modelling. In this paper, we analyze the behavior of three different viewpoint quality measures. The first one is a heuristic measure, the second on...

This paper proposes a novel online algorithm for nonnegative matrix factorization (NMF) based on the generalized Kullback-Leibler (KL) divergence criterion, aimed to overcome the high computation problem of large-scale data brought about by conventional batch NMF algorithms. It features stable updating the factors alternately for each new-coming observation, and provides an efficient solution f...