Automated machine learning (AutoML) systems commonly ensemble models post hoc to improve predictive performance, typically via greedy ensemble selection (GES). However, we believe that GES may not always be optimal, as it performs a simple deterministic greedy search. In this work, we introduce two novel population-based ensemble selection methods, QO-ES and QDO-ES, and compare them to GES. While QO-ES optimises solely for predictive performance, QDO-ES also considers the diversity of ensembles within the population, maintaining a diverse set of well-performing ensembles during optimisation based on ideas of quality diversity optimisation. The methods are evaluated using 71 classification datasets from the AutoML benchmark, demonstrating that QO-ES and QDO-ES often outrank GES, albeit only statistically significant on validation data. Our results further suggest that diversity can be beneficial for post hoc ensembling but also increases the risk of overfitting.

翻译：摘要：自动机器学习（AutoML）系统通常在事后通过集成模型来提升预测性能，常用方法为贪心集成选择（GES）。然而，我们认为GES可能并非始终最优，因其本质上是一种简单的确定性贪心搜索。本研究提出两种新型基于种群的集成选择方法——QO-ES与QDO-ES，并将其与GES进行对比。其中QO-ES仅以预测性能为优化目标，而QDO-ES则进一步考虑集成种群的多样性，基于质量多样性优化思想在优化过程中维持一组性能优异且差异化的集成模型。通过在AutoML基准测试中71个分类数据集上的评估表明，尽管仅在验证数据上具有统计显著性，QO-ES与QDO-ES的排序结果通常优于GES。进一步研究发现，多样性虽有助于提升后验集成效果，但同时也会增加过拟合风险。