We consider two entropy-based interior point methods that solve LP relaxations of MAP estimation in graphical models: (1) an entropy-regularization method and (2) a relative-entropy proximal method. Using the fact that relative-entropy is the Bregman distance induced by entropy, we show that the two approaches are actually equivalent. The purpose of this note is to show one connection between t...

Given iid samples drawn from a distribution with known parametric form, we propose the minimization of expected Bregman divergence to form Bayesian estimates of differential entropy and relative entropy, and derive such estimators for the uniform, Gaussian, Wishart, and inverse Wishart distributions. Additionally, formulas are given for a log gamma Bregman divergence and the differential entrop...

By revisiting the Kirchhoff’s Matrix-Tree Theorem, we give an exact formula for the number of spanning trees of a graph in terms of the quantum relative entropy between the maximally mixed state and another state specifically obtained from the graph. We use properties of the quantum relative entropy to prove tight bounds for the number of spanning trees in terms of basic parameters like degrees...

The measure-theoretic definition of Kullback-Leibler relative-entropy (KL-entropy) plays a basic role in the definitions of classical information measures. Entropy, mutual information and conditional forms of entropy can be expressed in terms of KL-entropy and hence properties of their measure-theoretic analogs will follow from those of measure-theoretic KL-entropy. These measure-theoretic defi...

In this paper I proved that quantum relative entropy D(ρ‖σ) can be asymptotically attained by Kullback Leibler divergences of probabilities given by a certain sequence of measurements. The sequence of measurements depends on ρ, but is independent of the choice of σ.

We determine the capacity of compound classicalquantum channels. As a consequence we obtain the capacity formula for the averaged classical-quantum channels. The capacity result for compound channels demonstrates, as in the classical setting, the existence of reliable universal classical-quantum codes in scenarios where the only a priori information about the channel used for the transmission o...

I propose the multi-mode squeezed thermal state based on the multimode pure entangled state. The correlation matrix of the state is characterized by two parameters. I then analysis the separable condition for this state, and calculating the relative entropy of the state with respect to the same kind of fully separable state in order to provide an upper bound of the relative entropy of entanglem...

The measure-theoretic definition of Kullback-Leibler relative-entropy (or simply KLentropy) plays a basic role in defining various classical information measures on general spaces. Entropy, mutual information and conditional forms of entropy can be expressed in terms of KL-entropy and hence properties of their measure-theoretic analogs will follow from those of measure-theoretic KL-entropy. The...

We consider entropy and relative entropy in field theory and establish relevant monotonicity properties with respect to the couplings. The relative entropy in a field theory with a hierarchy of renormalization-group fixed points ranks the fixed points, the lowest relative entropy being assigned to the highest multicritical point. We argue that as a consequence of a generalized H theorem Wilsoni...