This chapter describes how difficult statistical estimation problems can often be solved efficiently by means of the cross-entropy (CE) method. The CE method can be viewed as an adaptive importance sampling procedure that uses the cross-entropy or Kullback–Leibler divergence as a measure of closeness between two sampling distributions. The CE method is particularly useful for the estimation of ...

In general, conventional error correction for inverse synthetic aperture radarimaging consists of range alignment and phase adjustment, which compensate range shift and phase error, respectively. Minimum entropy-based methods have been proposed to realize range alignment and phase adjustment. However, it becomes challenging to align high-resolution profiles when strong noise presents, even usin...

Recently, different semantics for relational probabilistic conditionals and corresponding maximum entropy (ME) inference operators have been proposed. In this paper, we study the so-called aggregation semantics that covers both notions of a statistical and subjective view. The computation of its inference operator requires the calculation of the ME-distribution satisfying all probabilistic cond...

We study the class of self-similar probability density functions with finite mean and variance, which maximize Rényi’s entropy. The investigation is restricted in the Schwartz space S(R) and in the space of l-differentiable compactly supported functions C l c(R). Interestingly, the solutions of this optimization problem do not coincide with the solutions of the usual porous medium equation with...

The genetic algorithm is an optimization procedure motivated by biological evolution and is successfully applied to optimization problems in different areas. A statistical mechanics model for its dynamics is proposed based on the parent-child fitness correlation of the genetic operators, making it applicable to general fitness landscapes. It is compared to a recent model based on a maximum entr...

Bayesian Optimization Algorithm (BOA), a multivariate estimation of distribution algorithm, needs incorporating with efficiency enhancement techniques to be capable of solving difficult large-scale problems in a reliable and scalable manner. In this paper, we present a novel evaluation relaxation method which is based on the conditional entropy measurement. The concept of conditional entropy is...