This paper describes a new idea of finding the importance sampling density in rare events simulations: the MinxEnt method (shorthand for minimum cross-entropy). Some preliminary results show that the method might be very promising. 1 The minxent program Assume • X = (X1, . . . ,Xn) is a random vector (with values denoted by x); • h is the joint density function of X; • Sj(·) (j = 1, . . . , k) ...

In this paper we propose a method that learns to play Pac-Man. We define a set of high-level observation and action modules. Actions are temporally extended, and multiple action modules may be in effect concurrently. A decision of the agent is represented as a rule-based policy. For learning, we apply the cross-entropy method, a recent global optimization algorithm. The learned policies reached...

Based on the Markov model and the basic theory of information entropy, this paper puts forward a new method for optimizing the location of observation points in order to obtain more information from limited geological investigation. According to the existing data from observation points data, classification of tunnel geological lithology was performed, and various lithology distribution were de...

In this paper, a method is presented for the efficient estimation of rare-event (buffer overflow) probabilities in queueing networks using importance sampling. Unlike previously proposed change of measures, the one used here is not static, i.e., it depends on the buffer contents at each of the network nodes. The ‘optimal’ state-dependent change of measure is determined adaptively during the sim...

We present an information theoretic approach to stochastic optimal control problems that can be used to derive general sampling based optimization schemes. This new mathematical method is used to develop a sampling based model predictive control algorithm. We apply this information theoretic model predictive control (IT-MPC) scheme to the task of aggressive autonomous driving around a dirt test...

Nonparametric density estimation aims to determine the sparsest model that explains a given set of empirical data and which uses as few assumptions as possible. Many of the currently existing methods do not provide a sparse solution to the problem and rely on asymptotic approximations. In this paper we describe a framework for density estimation which uses information-theoretic measures of mode...

The cross-entropy method is an efficient and general optimization algorithm. However, its applicability in reinforcement learning (RL) seems to be limited because it often converges to suboptimal policies. We apply noise for preventing early convergence of the cross-entropy method, using Tetris, a computer game, for demonstration. The resulting policy outperforms previous RL algorithms by almos...

We present new theoretical convergence results on the Cross-Entropy method for discrete optimization. Our primary contribution is to show that a popular implementation of the Cross-Entropy method converges, and finds an optimal solution with probability arbitrarily close to 1. We also give necessary conditions and sufficient conditions under which an optimal solution is generated eventually wit...

Cross-entropy optimization (CE) has proven to be a powerful tool for search in control environments. In the basic scheme, a distribution over proposed solutions is repeatedly adapted by evaluating a sample of solutions and refocusing the distribution on a percentage of those with the highest scores. We show that, in the kind of noisy evaluation environments that are common in decisionmaking dom...

The buffer allocation problem (BAP) is a well-known difficult problem in the design of production lines. We present a stochastic algorithm for solving the BAP, based on the cross-entropy method, a new paradigm for stochastic optimization. The algorithm involves the following iterative steps: (a) the generation of buffer allocations according to a certain random mechanism, followed by (b) the mo...