Meta-learning aims to extract useful inductive biases from a set of related datasets. In Bayesian meta-learning, this is typically achieved by constructing a prior distribution over neural network parameters. However, specifying families of computationally viable prior distributions over the high-dimensional neural network parameters is difficult. As a result, existing approaches resort to meta-learning restrictive diagonal Gaussian priors, severely limiting their expressiveness and performance. To circumvent these issues, we approach meta-learning through the lens of functional Bayesian neural network inference, which views the prior as a stochastic process and performs inference in the function space. Specifically, we view the meta-training tasks as samples from the data-generating process and formalize meta-learning as empirically estimating the law of this stochastic process. Our approach can seamlessly acquire and represent complex prior knowledge by meta-learning the score function of the data-generating process marginals instead of parameter space priors. In a comprehensive benchmark, we demonstrate that our method achieves state-of-the-art performance in terms of predictive accuracy and substantial improvements in the quality of uncertainty estimates.

翻译：元学习旨在从一组相关数据集中提取有用的归纳偏置。在贝叶斯元学习中，这通常通过构建神经网络参数的先验分布来实现。然而，在高维神经网络参数空间上指定计算上可行的先验分布族是困难的。因此，现有方法往往局限于元学习约束性的对角高斯先验，严重限制了其表达能力和性能。为解决这些问题，我们从函数贝叶斯神经网络推理的角度出发处理元学习，将先验视为一个随机过程，并在函数空间中进行推理。具体而言，我们将元训练任务视为数据生成过程的样本，并将元学习形式化为对该随机过程定律的经验估计。我们的方法能够通过元学习数据生成过程边际的分数函数（而非参数空间先验）来无缝获取并表征复杂的先验知识。在综合基准测试中，我们证明该方法在预测准确性方面达到了最先进的性能，并在不确定性估计质量上取得了显著提升。