Optimizing ensemble weights and hyperparameters of machine learning models for regression problems

Aggregating multiple learners through an ensemble of models aim to make better predictions by capturing the underlying distribution data more accurately. Different ensembling methods, such as bagging, boosting, and stacking/blending, have been studied adopted extensively in research practice. While bagging boosting focus on reducing variance bias, respectively, stacking approaches target both finding optimal way combine base learners. In with weighted average, ensembles are created from averages It is known that tuning hyperparameters each learner inside weight optimization process can produce performing ensembles. To this end, optimization-based nested algorithm considers well weights (Generalized Weighted Ensemble Internally Tuned Hyperparameters (GEM-ITH)) designed. Besides, Bayesian search was used speed-up optimizing a heuristic implemented generate diverse well-performing The shown be generalizable real sets analyses ten publicly available sets.