This paper introduces a modular framework for Mixed-variable and Combinatorial Bayesian Optimization (MCBO) to address the lack of systematic benchmarking and standardized evaluation in the field. Current MCBO papers often introduce non-diverse or non-standard benchmarks to evaluate their methods, impeding the proper assessment of different MCBO primitives and their combinations. Additionally, papers introducing a solution for a single MCBO primitive often omit benchmarking against baselines that utilize the same methods for the remaining primitives. This omission is primarily due to the significant implementation overhead involved, resulting in a lack of controlled assessments and an inability to showcase the merits of a contribution effectively. To overcome these challenges, our proposed framework enables an effortless combination of Bayesian Optimization components, and provides a diverse set of synthetic and real-world benchmarking tasks. Leveraging this flexibility, we implement 47 novel MCBO algorithms and benchmark them against seven existing MCBO solvers and five standard black-box optimization algorithms on ten tasks, conducting over 4000 experiments. Our findings reveal a superior combination of MCBO primitives outperforming existing approaches and illustrate the significance of model fit and the use of a trust region. We make our MCBO library available under the MIT license at \url{https://github.com/huawei-noah/HEBO/tree/master/MCBO}.

翻译：本文提出了一个模块化的组合变量与混合变量贝叶斯优化（MCBO）框架，旨在解决该领域缺乏系统性基准测试与标准化评估的问题。当前MCBO论文常采用非多样化或非标准化的基准方法进行性能评估，这阻碍了对不同MCBO原语及其组合方式的合理评价。此外，针对单一MCBO原语提出解决方案的论文往往忽略与其他采用相同方法处理剩余原语的基线进行对比。这种缺陷主要源于巨大的实现开销，导致缺乏受控评估，无法有效展示研究成果的价值。为应对这些挑战，我们的框架支持贝叶斯优化组件的无缝组合，并提供多样化的合成与真实世界基准测试任务。利用此灵活性，我们实现了47种新型MCBO算法，并在十项任务中将其与七种现有MCBO求解器及五种标准黑箱优化算法进行基准测试，共开展超过4000次实验。研究发现存在优于现有方法的MCBO原语组合，并揭示了模型拟合效果以及信任区域使用的重要性。我们已在MIT许可协议下开放MCBO库：\url{https://github.com/huawei-noah/HEBO/tree/master/MCBO}。