This paper introduces a modular framework for Mixed-variable and Combinatorial Bayesian Optimization (MCBO) to address the lack of systematic benchmarking and standardized evaluation in the field. Current MCBO papers often introduce non-diverse or non-standard benchmarks to evaluate their methods, impeding the proper assessment of different MCBO primitives and their combinations. Additionally, papers introducing a solution for a single MCBO primitive often omit benchmarking against baselines that utilize the same methods for the remaining primitives. This omission is primarily due to the significant implementation overhead involved, resulting in a lack of controlled assessments and an inability to showcase the merits of a contribution effectively. To overcome these challenges, our proposed framework enables an effortless combination of Bayesian Optimization components, and provides a diverse set of synthetic and real-world benchmarking tasks. Leveraging this flexibility, we implement 47 novel MCBO algorithms and benchmark them against seven existing MCBO solvers and five standard black-box optimization algorithms on ten tasks, conducting over 4000 experiments. Our findings reveal a superior combination of MCBO primitives outperforming existing approaches and illustrate the significance of model fit and the use of a trust region. We make our MCBO library available under the MIT license at \url{https://github.com/huawei-noah/HEBO/tree/master/MCBO}.

翻译：本文介绍了一个面向混合变量与组合贝叶斯优化（MCBO）的模块化框架，旨在解决该领域缺乏系统性基准测试与标准化评估的问题。现有MCBO论文常采用非多样化或非标准化的基准来评估其方法，导致不同MCBO原语及其组合难以得到合理评估。此外，针对单一MCBO原语提出解决方案的论文往往省略与采用相同方法处理其余原语的基线方法进行对比——这主要源于巨大的实现开销，进而造成控制性评估缺失，无法有效展现研究成果的优势。为应对上述挑战，本框架支持贝叶斯优化组件的灵活组合，并提供多样化合成任务与真实世界基准测试任务。借助这一灵活性，我们实现了47种新型MCBO算法，并在十个任务上将其与七种现有MCBO求解器及五种标准黑箱优化算法进行对比，开展了超过4000次实验。研究结果揭示了优于现有方案的MCBO原语组合方案，同时阐明了模型拟合质量与信任域使用的重要性。我们将MCBO库以MIT许可证开源发布于 \url{https://github.com/huawei-noah/HEBO/tree/master/MCBO}。