Learning the kernel parameters for Gaussian processes is often the computational bottleneck in applications such as online learning, Bayesian optimization, or active learning. Amortizing parameter inference over different datasets is a promising approach to dramatically speed up training time. However, existing methods restrict the amortized inference procedure to a fixed kernel structure. The amortization network must be redesigned manually and trained again in case a different kernel is employed, which leads to a large overhead in design time and training time. We propose amortizing kernel parameter inference over a complete kernel-structure-family rather than a fixed kernel structure. We do that via defining an amortization network over pairs of datasets and kernel structures. This enables fast kernel inference for each element in the kernel family without retraining the amortization network. As a by-product, our amortization network is able to do fast ensembling over kernel structures. In our experiments, we show drastically reduced inference time combined with competitive test performance for a large set of kernels and datasets.

翻译：学习高斯过程的核参数通常是诸如在线学习、贝叶斯优化或主动学习等应用中的计算瓶颈。在不同数据集上摊销参数推断是大幅缩短训练时间的一种有前景的方法。然而，现有方法将摊销推断过程限制在固定的核结构上。若采用不同的核，则必须手动重新设计摊销网络并重新训练，导致设计和训练时间开销巨大。我们提出在完整的核结构族（而非固定的核结构）上摊销核参数推断。通过定义一个作用于数据集与核结构对的摊销网络来实现这一点。这使得无需重新训练摊销网络即可对核族中的每个元素进行快速核推断。作为副产品，我们的摊销网络能够实现核结构的快速集成。在实验中，我们展示了对于大量核和数据集，推断时间大幅减少的同时测试性能具有竞争力。