We perform Bayesian optimization using a Gaussian process perspective on Bayesian Additive Regression Trees (BART). Our BART Kernel (BARK) uses tree agreement to define a posterior over piecewise-constant functions, and we explore the space of tree kernels using a Markov chain Monte Carlo approach. Where BART only samples functions, the resulting BARK model obtains samples of Gaussian processes defining distributions over functions, which allow us to build acquisition functions for Bayesian optimization. Our tree-based approach enables global optimization over the surrogate, even for mixed-feature spaces. Moreover, where many previous tree-based kernels provide uncertainty quantification over function values, our sampling scheme captures uncertainty over the tree structure itself. Our experiments show the strong performance of BARK on both synthetic and applied benchmarks, due to the combination of our fully Bayesian surrogate and the optimization procedure.

翻译：我们采用高斯过程视角对贝叶斯加性回归树（BART）进行贝叶斯优化。所提出的BART核（BARK）通过树结构一致性定义分段常数函数的后验分布，并采用马尔可夫链蒙特卡洛方法探索树核空间。传统BART仅对函数进行采样，而BARK模型能够获得定义函数分布的高斯过程样本，从而构建适用于贝叶斯优化的采集函数。这种基于树的方法使得代理模型能够实现全局优化，即使在混合特征空间中亦能适用。此外，相较于以往多数树核仅能对函数值进行不确定性量化，我们的采样方案能够捕捉树结构本身的不确定性。实验表明，基于全贝叶斯代理模型与优化流程的结合，BARK在合成基准测试与应用基准测试中均表现出卓越性能。