| We describe an algorithm to eeciently compute maximum entropy densities, i.e. densities maximizing the Shannon entropy ? R p(x) log p(x)dx under a set of constraints Eg i (x)] = c i , i = 1;: : : ; n: Our method is based on an algorithm by Zellner and Highheld, which has been found not to converge under a variety of circumstances. To demonstrate that our method overcomes these diiculties, we ...

We show that the Jaynes principle is indeed a proper inference scheme when applied to compound systems and will correctly produce the entangled maximum entropy states compatible with appropriate data. This is accomplished by including the dispersion of the entanglement along with its mean value as constraints and an application of the uncertainty principle. We also construct a "thermodynamic-li...

This paper is a contribution to the discussion on the relation between thermodynamics and economic theory. With respect to thermodynamic constraints on the economy, there are two diametrically opposite positions in this discussion. One claims that the constraints are insignificant (‘of no immediate practical importance for modelling’) and in the intermediate run, do not limit economic activity ...

This work presents PESMOC, Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints, an information-based strategy for the simultaneous optimization of multiple expensive-to-evaluate black-box functions under the presence of several constraints. PESMOC can hence be used to solve a wide range of optimization problems. Iteratively, PESMOC chooses an input location on w...

We discuss a one-parameter family of generalized cross entropy between two distributions with the power index, called the projective power entropy. The cross entropy is essentially reduced to the Tsallis entropy if two distributions are taken to be equal. Statistical and probabilistic properties associated with the projective power entropy are extensively investigated including a characterizati...

Maximum entropy framework proved to be expressive and powerful for the statistical language modelling, but it suffers from the computational expensiveness of the model building. The iterative scaling algorithm that is used for the parameter estimation is computationally expensive while the feature selection process requires to estimate parameters of the model for many candidate features many ti...

An information theoretic approach is used to examine the magnitude and origin of associations among amino acid sites in the basic helix-loop-helix (bHLH) family of transcription factors. Entropy and mutual information values are used to summarize the variability and covariability of amino acids comprising the bHLH domain for 242 sequences. When these quantitative measures are integrated with cr...

| The problem of initial probability assignment consistent with the available information about a probabilis-tic system is called a direct problem. Jaynes' maximum en-tropy principle (MaxEnt) provides a method for solving direct problems when the available information is in the form of moment constraints. On the other hand, given a probability distribution, the problem of nding a set of constra...