Greed Is Good: Exploration and Exploitation Trade-offs in Bayesian Optimisation
نویسندگان
چکیده
The performance of acquisition functions for Bayesian optimisation to locate the global optimum continuous is investigated in terms Pareto front between exploration and exploitation. We show that Expected Improvement (EI) Upper Confidence Bound (UCB) always select solutions be expensively evaluated on front, but Probability not guaranteed do so Weighted does only a restricted range weights. introduce two novel -greedy functions. Extensive empirical evaluation these together with random search, purely exploratory, exploitative search 10 benchmark problems 1 dimensions shows algorithms are generally at least as effective conventional (e.g., EI UCB), particularly limited budget. In higher dimensions, approaches shown have improved over approaches. These results borne out real-world computational fluid dynamics problem robotics active learning problem. Our analysis experiments suggest most strategy, mostly greedy, occasionally selecting exploratory solution.
منابع مشابه
Learning and innovation: Exploitation and exploration trade-offs☆
a r t i c l e i n f o This paper examines the relationship between learning and innovation outcomes, focusing on the trade-off between exploitation and exploration in learning and innovation. The study identifies two types of learning and two outcomes of innovation. Exploitation and exploration in learning are inversely associated with innovation rates and impact. While exploitative, localized ...
متن کاملUsing Confidence Bounds for Exploitation-Exploration Trade-offs
We show how a standard tool from statistics — namely confidence bounds — can be used to elegantly deal with situations which exhibit an exploitation-exploration trade-off. Our technique for designing and analyzing algorithms for such situations is general and can be applied when an algorithm has to make exploitation-versus-exploration decisions based on uncertain information provided by a rando...
متن کاملPAC-Bayesian Analysis of the Exploration-Exploitation Trade-off
We develop a coherent framework for integrative simultaneous analysis of the explorationexploitation and model order selection tradeoffs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a combination is also of independent interest for studies of multiple simultaneously evolvi...
متن کاملGradient Algorithms for Exploration/Exploitation Trade-Offs: Global and Local Variants
Gradient-following algorithms are deployed for efficient adaptation of exploration parameters in temporal-difference learning with discrete action spaces. Global and local variants are evaluated in discrete and continuous state spaces. The global variant is memory efficient in terms of requiring exploratory data only for starting states. In contrast, the local variant requires exploratory data ...
متن کاملA Rational-Ecological Approach to the Exploration/Exploitation Trade-offs Bounded Rationality and Suboptimal Performance
After years of research, it is embarrassing how little we understand the underlying processes of adaptation: just look at how difficult it is to build a robot that learns to navigate in a new environment or to teach someone to master a second language. It is amazing how seagulls and vultures have learned to be landfill scavengers in the last century and be able to sort through human garbage to ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: ACM transactions on evolutionary learning
سال: 2021
ISSN: ['2688-3007', '2688-299X']
DOI: https://doi.org/10.1145/3425501