Bayesian Optimization (BO) is an effective approach for global optimization of black-box functions when function evaluations are expensive. Most prior works use Gaussian processes to model the black-box function, however, the use of kernels in Gaussian processes leads to two problems: first, the kernel-based methods scale poorly with the number of data points and second, kernel methods are usually not effective on complex structured high dimensional data due to curse of dimensionality. Therefore, we propose a novel black-box optimization algorithm where the black-box function is modeled using a neural network. Our algorithm does not need a Bayesian neural network to estimate predictive uncertainty and is therefore computationally favorable. We analyze the theoretical behavior of our algorithm in terms of regret bound using advances in NTK theory showing its efficient convergence. We perform experiments with both synthetic and real-world optimization tasks and show that our algorithm is more sample efficient compared to existing methods.

翻译：贝叶斯优化（BO）是函数评估代价高昂时，对黑箱函数进行全局优化的有效方法。以往大多数研究使用高斯过程对黑箱函数建模，但高斯过程中的核函数会导致两个问题：第一，基于核的方法随数据点数量增加而扩展性较差；第二，由于维度灾难，核方法通常难以有效处理复杂的结构化高维数据。为此，我们提出一种新颖的黑箱优化算法，其中黑箱函数通过神经网络进行建模。该算法无需使用贝叶斯神经网络来估计预测不确定性，因此在计算上更为高效。我们借助神经正切核（NTK）理论的最新进展，从遗憾界限角度分析了算法的理论行为，证明了其高效收敛性。我们在合成数据集和真实优化任务上开展实验，结果表明，与现有方法相比，我们的算法在样本效率上更优。