Bayesian Optimization (BO) is a powerful tool for black-box optimization, but its application to high-dimensional permutation spaces is severely limited by the challenge of defining scalable representations. The current state-of-the-art BO approach for permutation spaces relies on an exhaustive $Ω(n^2)$ pairwise comparison, inducing a dense representation that is impractical for large-scale permutations. To break this barrier, we introduce a novel framework for generating efficient permutation representations via kernel functions derived from sorting algorithms. Within this framework, the Mallows kernel can be viewed as a special instance derived from enumeration sort. Further, we introduce the \textbf{Merge Kernel} , which leverages the divide-and-conquer structure of merge sort to produce a compact, $Θ(n\log n)$ to achieve the lowest possible complexity with no information loss and effectively capture permutation structure. Our central thesis is that the Merge Kernel performs competitively with the Mallows kernel in low-dimensional settings, but significantly outperforms it in both optimization performance and computational efficiency as the dimension $n$ grows. Extensive evaluations on various permutation optimization benchmarks confirm our hypothesis, demonstrating that the Merge Kernel provides a scalable and more effective solution for Bayesian optimization in high-dimensional permutation spaces, thereby unlocking the potential for tackling previously intractable problems such as large-scale feature ordering and combinatorial neural architecture search.
翻译:贝叶斯优化(BO)是黑箱优化的重要工具,但在高维置换空间中,其应用因可扩展表示的挑战而严重受限。当前针对置换空间的最先进BO方法依赖于穷举的$Ω(n^2)$成对比较,导致密集表示难以应对大规模置换。为突破这一瓶颈,我们提出了一种新框架,通过从排序算法推导核函数来生成高效置换表示。在该框架下,Mallows核可被视为枚举排序导出的特例。进一步,我们引入**归并核**,利用归并排序的分治结构产生紧凑的$Θ(n \log n)$复杂度表示,在无信息损失的前提下实现理论最低复杂度,并有效捕获置换结构。我们的核心论点是:在低维场景中,归并核与Mallows核性能相当;但随着维度$n$增大,归并核在优化性能和计算效率上均显著超越后者。在多种置换优化基准上的广泛评估验证了我们的假设,证明归并核为高维置换空间中的贝叶斯优化提供了可扩展且更高效的解决方案,从而为解决大规模特征排序和组合神经架构搜索等此前难以处理的问题开辟了可能。