Maximum Entropy (MaxEnt) language models are powerful models that can incorporate linguistic and non-linguistic contextual signals in a unified framework with a convex loss. MaxEnt models also have the advantage of scaling to large model and training data sizes We present the following two contributions to MaxEnt training: (1) By leveraging smaller amounts of transcribed data, we demonstrate th...

There are several mystifications and a couple of mysteries pertinent to MaxEnt. The mystifications, pitfalls and traps are set up mainly by an unfortunate formulation of Jaynes’ die problem, the cause célèbre of MaxEnt. After discussing the mystifications a new formulation of the problem is proposed. Then we turn to the mysteries. An answer to the recurring question ’Just what are we accomplish...

In this paper, we propose an improved principal component analysis based on maximum entropy (MaxEnt) preservation, called MaxEnt-PCA, which is derived from a Parzen window estimation of Renyi’s quadratic entropy. Instead of minimizing the reconstruction error either based on L2-norm or L1-norm, the MaxEnt-PCA attempts to preserve as much as possible the uncertainty information of the data measu...

This paper shows that: (a) given a mechanical system described by a set of independent coordinates in configuration space, (b) given an initial state of specified initial coordinates, and (c) given a situation in which the system can follow any one of a set of different possible paths with a pertinent probability pi , then the average path (defined as the weighted average positions and correspo...

BACKGROUND Plague, caused by the bacterium Yersinia pestis, is a public and wildlife health concern in California and the western United States. This study explores the spatial characteristics of positive plague samples in California and tests Maxent, a machine-learning method that can be used to develop niche-based models from presence-only data, for mapping the potential distribution of plagu...

Probabilistic models for binary spike patterns provide a powerful tool for understanding the statistical dependencies in large-scale neural recordings. Maximum entropy (or “maxent”) models, which seek to explain dependencies in terms of low-order interactions between neurons, have enjoyed remarkable success in modeling such patterns, particularly for small groups of neurons. However, these mode...

Climate warming is one of the most important threats to biodiversity. Ectothermic organisms such as amphibians and reptiles are especially vulnerable as climatic conditions affect them directly. Ecological niche models (ENMs) are increasingly popular in ecological studies, but several drawbacks exist, including the limited ability to account for the dispersal potential of the species. In this s...

We propose a new kernel-based data transformation technique. It is founded on the principle of maximum entropy (MaxEnt) preservation, hence named kernel MaxEnt. The key measure is Renyi’s entropy estimated via Parzen windowing. We show that kernel MaxEnt is based on eigenvectors, and is in that sense similar to kernel PCA, but may produce strikingly different transformed data sets. An enhanced ...

Residual dipolar couplings (RDC) provide important global restraints for accurate structure determination by NMR. We show that nonuniform sampling in combination with maximum entropy reconstruction (MaxEnt) is a promising strategy for accelerating and potentially enhancing the acquisition of RDC spectra. Using MaxEnt-processed spectra of nonuniformly sampled data sets that are reduced up to one...

In this paper, we propose a Robust Discriminant Analysis based on maximum entropy (MaxEnt) criterion (MaxEnt-RDA), which is derived from a nonparametric estimate of Renyi’s quadratic entropy. MaxEnt-RDA uses entropy as both objective and constraints; thus the structural information of classes is preserved while information loss is minimized. It is a natural extension of LDA from Gaussian assump...