Meta-learning aims to extract useful inductive biases from a set of related datasets. In Bayesian meta-learning, this is typically achieved by constructing a prior distribution over neural network parameters. However, specifying families of computationally viable prior distributions over the high-dimensional neural network parameters is difficult. As a result, existing approaches resort to meta-learning restrictive diagonal Gaussian priors, severely limiting their expressiveness and performance. To circumvent these issues, we approach meta-learning through the lens of functional Bayesian neural network inference, which views the prior as a stochastic process and performs inference in the function space. Specifically, we view the meta-training tasks as samples from the data-generating process and formalize meta-learning as empirically estimating the law of this stochastic process. Our approach can seamlessly acquire and represent complex prior knowledge by meta-learning the score function of the data-generating process marginals instead of parameter space priors. In a comprehensive benchmark, we demonstrate that our method achieves state-of-the-art performance in terms of predictive accuracy and substantial improvements in the quality of uncertainty estimates.

翻译：元学习旨在从一组相关数据集中提取有用的归纳偏置。在贝叶斯元学习中，这通常通过构建神经网络参数上的先验分布来实现。然而，在高维神经网络参数空间上指定计算可行的先验分布族十分困难。因此，现有方法只能元学习具有限制性的对角高斯先验，严重制约了其表达能力和性能。为解决这些问题，我们通过函数贝叶斯神经网络推理的视角来处理元学习，将先验视为随机过程并在函数空间中进行推理。具体而言，我们将元训练任务视为数据生成过程中的样本，并将元学习形式化为对该随机过程律的实证估计。我们的方法通过元学习数据生成过程边际的分数函数（而非参数空间先验），能够无缝获取和表示复杂的先验知识。在全面的基准测试中，我们证明该方法在预测准确性上达到了最先进水平，并在不确定性估计质量上取得了显著提升。