— In this paper, the performances of the quasi-Newton BFGS algorithm, the NEWUOA derivative free optimizer, the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), the Differential Evolution (DE) algorithm and Particle Swarm Optimizers (PSO) are compared experimentally on benchmark functions reflecting important challenges encountered in real-world optimization problems. Dependence of the...

This thesis considers variable metrics in the context of stochastic, function-value free optimization in continuous search spaces. We argue that the choice of a (variable) metric or equivalently the choice of a coordinate system can be decoupled from the underlying optimization procedure. An adaptive encoding procedure is presented, that is in principle applicable to any optimization procedure,...

Hybridization in context to Evolutionary Computation (EC) aims at combining the operators and methodologies from different EC paradigms to form a single algorithm that may enjoy a statistically superior performance on a wide variety of optimization problems. In this article we propose an efficient hybrid evolutionary algorithm that embeds the difference vector-based mutation scheme, the crossov...

Evolution Strategies, Evolutionary Algorithms based on Gaussian mutation and deterministic selection, are today considered the best choice as far as parameter optimization is concerned. However, there are multiple ways to tune the covariance matrix of the Gaussian mutation. After reviewing the state of the art in covariance matrix adaptation, a new approach is proposed, in which the covariance ...

The covariance matrix adaptation evolution strategy (CMA-ES) is arguably one of the most powerful real-valued derivative-free optimization algorithms, finding many applications in machine learning. The CMA-ES is a Monte Carlo method, sampling from a sequence of multi-variate Gaussian distributions. Given the function values at the sampled points, updating and storing the covariance matrix domin...

This paper compares several Gaussian-processbased surrogate modeling methods applied to black-box optimization by means of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), which is considered state-of-the-art in the area of continuous black-box optimization. Among the compared methods are the Modelassisted CMA-ES, the Robust Kriging Metamodel CMAES, and the Surrogate CMA-ES. In add...

The Steady State variants of the Multi-Objective Covariance Matrix Adaptation Evolution Strategy (SS-MO-CMA-ES) generate one offspring from a uniformly selected parent. Some other parental selection operators for SS-MO-CMA-ES are investigated in this paper. These operators involve the definition of multi-objective rewards, estimating the expectation of the offspring survival and its Hypervolume...

Following the introduction of two niching methods within Evolution Strategies (ES), which have been presented recently and have been successfully applied to theoretical high-dimensional test functions, as well as to a real-life high-dimensional physics problem, the purpose of this study is to address the so-called niche radius problem. A new concept of adaptive individual niche radius, introduc...

The covariance matrix adaptation evolution strategy (CMA-ES) is one of the most powerful evolutionary algorithms for real-valued single-objective optimization. Here a variant of the CMA-ES for multi-objective optimization (MOO) is developed. First a single-objective, elitist CMA-ES using plus-selection and step size control based on a success rule is introduced. This algorithm is compared to th...

The covariance matrix adaptation evolution strategy (CMA-ES) is one of the most powerful evolutionary algorithms for real-valued single-objective optimization. In this paper, we develop a variant of the CMA-ES for multi-objective optimization (MOO). We first introduce a single-objective, elitist CMA-ES using plus-selection and step size control based on a success rule. This algorithm is compare...