Automated machine learning (AutoML) systems commonly ensemble models post hoc to improve predictive performance, typically via greedy ensemble selection (GES). However, we believe that GES may not always be optimal, as it performs a simple deterministic greedy search. In this work, we introduce two novel population-based ensemble selection methods, QO-ES and QDO-ES, and compare them to GES. While QO-ES optimises solely for predictive performance, QDO-ES also considers the diversity of ensembles within the population, maintaining a diverse set of well-performing ensembles during optimisation based on ideas of quality diversity optimisation. The methods are evaluated using 71 classification datasets from the AutoML benchmark, demonstrating that QO-ES and QDO-ES often outrank GES, albeit only statistically significant on validation data. Our results further suggest that diversity can be beneficial for post hoc ensembling but also increases the risk of overfitting.

翻译：自动化机器学习（AutoML）系统通常在后验阶段通过集成模型来提升预测性能，常用方法为贪心集成选择（GES）。然而，我们认为GES并非始终最优，因其执行的是简单的确定性贪心搜索。本文提出了两种新颖的基于种群的集成选择方法——QO-ES与QDO-ES，并与GES进行对比。其中，QO-ES仅优化预测性能，而QDO-ES在此基础上进一步考虑集成种群的多样性，基于质量多样性优化思想在优化过程中维持一组性能优异且多样化的集成。我们使用AutoML基准测试中的71个分类数据集对方法进行评估，结果表明QO-ES和QDO-ES通常优于GES，尽管仅在验证数据上具有统计显著性。进一步的结果表明，多样性对后验集成具有潜在益处，但也可能增加过拟合风险。