The paper proposes a new message passing algorithm for cycle-free factor graphs. The proposed ”entropy message passing” (EMP) algorithm may be viewed as sumproduct message passing over the entropy semiring, which has previously appeared in automata theory. The primary use of EMP is to compute the entropy of a model. However, EMP can also be used to compute expressions that appear in expectation...

The equilibrium state of a relativistic gas has been calculated based on the maximum entropy principle. Though the relativistic equilibrium state was long believed to be the Jüttner distribution, a number of papers have been published in recent years proposing alternative equilibrium states. However, some of these papers do not pay enough attention to the covariance of distribution functions, r...

This paper proposes a new method for blind inversion of a monotonic nonlinear map applied to a sum of random variables. Such kinds of mixtures of random variables are found in source separation and Wiener system inversion problems, for example. The importance of our proposed method is based on the fact that it permits to decouple the estimation of the nonlinear part (nonlinear compensation) fro...

We present a new technique of annealing the EM algorithm to allow for its tractable application to fitting models which include graph structures like assignments. The method, which can be generally used to sparsify dependence models, is applied to solve the assignment problem for the shared-resources Gaussian mixture model (e.g. [4], [5],[9]), and is compared to (and contrasted to) the widely u...

It is often assumed that maximization of the statistical entropy cannot account for the spontaneous emergence of currents or compositional heterogeneity associated with them, and that other principles involving entropy production (its maximization or minimization) must be invoked to explain the emergence and robustness of the order in driven dissipative systems. I show for a class of simple mod...

Cardinality estimation is the process of estimating the number of tuples returned by a query. In relational database query optimization, cardinality estimates are key statistics used by the optimizer to choose an (expected) lowest cost plan. As a result of the importance of the problem, there are many sources of statistical information available to the optimizer, e.g., query feedback records [S...

The expectation maximization (EM) algorithm is extensively used for tomographic image reconstruction based on Positron Emission Tomography (PET) modality. The EM algorithm gives good reconstructed images compared to those created by deterministic methods such as Filtered Back Projection (FBP) and Convolution Back projection (CBP). However, the computational complexity of EM-based algorithm is h...

Information measures arise in many disciplines, including forecasting (where scoring rules are used to provide incentives for probability estimation), signal processing (where information gain is measured in physical units of relative entropy), decision analysis (where new information can lead to improved decisions), and finance (where investors optimize portfolios based on their private inform...