This paper reviews the principle of Minimum Description Length (MDL) for problems of model selection. By viewing statistical modeling as a means of generating descriptions of observed data, the MDL framework discriminates between competing models based on the complexity of each description. This approach began with Kolmogorov’s theory of algorithmic complexity, matured in the literature on info...

Estimation of Distribution Algorithms (EDA) have been proposed as an extension of genetic algorithms. In this paper we explain the relationship of EDA to algorithms developed in statistics, artificial intelligence, and statistical physics. The major design issues are discussed within a general interdisciplinary framework. It is shown that maximum entropy approximations play a crucial role. All ...

Most analysts feel about scalar valued, linear heat flows the way they do about their grandparents: they are stable, reliable, but dull. To a large extent, the origin of their stability is the minimum principle, which says that the flow preserves non-negativity. In this lecture, I will describe some examples of what can happen when one introduces boundary conditions which destroy the minimum pr...

Zhi-Cheng Yang,1 Armin Rahmani,2, 3 Alireza Shabani,4 Hartmut Neven,4 and Claudio Chamon1 1Physics Department, Boston University, Boston, Massachusetts 02215, USA 2Department of Physics and Astronomy and Quantum Matter Institute, University of British Columbia, Vancouver, British Columbia, Canada V6T 1Z4 3Department of Physics and Astronomy, Western Washington University, 516 High Street, Belli...

The idea of inferring phylogenies by selecting trees that minimize the total tree length can be traced back to the 19th century and the mathematician Jakob Steiner. It complies with Occam’s principle of scientiŽc inference, which essentially maintains that simpler explanations are preferable to more complicated ones and that ad hoc explanations should be avoided. Parsimony methods, which infer ...

Probabilistic networks can be constructed from a database of cases by selecting a network that has highest quality with respect to this database according to a given measure. A new measure is presented for this purpose based on a minimum description length (MDL) approach. This measure is compared with a commonly used measure based on a Bayesian approach both from a theoretical and an experiment...

As science enters the 21st century, the study of complex systems is increasingly gaining importance. Complex systems are characterized by an extremely large number of units (many millions) which cooperate to produce complex observable behavior. Perhaps the most striking example of such a system is the human brain, a network of 1010 nerve cells whose simultaneous activity results in all human fu...

Dynamics of many complex systems can be described by replicator equations (RE). Here we present an effective method for solving a wide class of RE based on reduction theorems for models of inhomogeneous communities. The solutions of the RE minimize the discrimination information of the initial and current distributions at each point of the system trajectory, not only at the equilibrium, under t...

Hybrid optimal control problems are studied for systems where autonomous and controlled state jumps are allowed at the switching instants and in addition to running costs, switching between discrete states incurs costs. A key aspect of the analysis is the relationship between the Hamiltonian and the adjoint process before and after the switching instants as well as the relationship between adjo...

For a variational inequality problem (VIP) with a psudomonotone mapping F on its solution set C∗, we give equivalent statements for C∗ to be determined by the zeroes Γ(c∗) of the primal gap function of VIP, where c∗ ∈ C∗. One sufficient condition is also presented in terms of weaker sharpness of C∗. With the psudomonotonicity∗ of F onC∗ being characterized, C∗ turns out to coincide with the zer...