We present a new approach to inferring a probability distribution which is incompletely specified by a number of linear constraints. We argue that the currently most popular approach of entropy maximization depends on a “constraints as knowledge” interpretation of the constraints, and that a different “constraints as data” perspective leads to a completely different type of inference procedures...

One gives a recursive algorithm for the computation of the first and second order derivatives of the entropy of a periodic autoregressive process with respect to the autocovariances. It is an extension of the periodic LevinsonDurbin algorithm. The algorithm has been developed for use at one of the steps of an entropy maximization method developed by the authors. Numerical examples of entropy ma...

Submodular function maximization is one of the key problems that arise in many machine learning tasks. Greedy selection algorithms are the proven choice to solve such problems, where prior theoretical work guarantees (1 − 1/e) approximation ratio. However, it has been empirically observed that greedy selection provides almost optimal solutions in practice. The main goal of this paper is to expl...

The binomial and the Poisson distributions are shown to be maximum entropy distributions of suitably defined sets. Poisson’s law is considered as a case of entropy maximization, and also convergence in information divergence is established.

Water engineering is an amalgam of engineering (e.g., hydraulics, hydrology, irrigation, ecosystems, environment, water resources) and non-engineering (e.g., social, economic, political) aspects that are needed for planning, designing and managing water systems. These aspects and the associated issues have been dealt with in the literature using different techniques that are based on different ...

A principle of information-entropy maximization is introduced in order to characterize the optimal representation of an arbitrarily varying quantity by a neural output confined to a finite interval. We then study the conditions under which a neuron can effectively fulfil the requirements imposed by this information-theoretic optimal principle. We show that this can be achieved with the natural ...

There are two distinct approaches for deriving the canonical ensemble. The canonical ensemble either follows as a special limit of the microcanonical ensemble or alternatively follows from the maximum entropy principle. We show the equivalence of these two approaches by applying the maximum entropy formulation to a closed universe consisting of an open system plus bath. We show that the target ...