Neural network approaches for meta-learning distributions over functions have desirable properties such as increased flexibility and a reduced complexity of inference. Building on the successes of denoising diffusion models for generative modelling, we propose Neural Diffusion Processes (NDPs), a novel approach that learns to sample from a rich distribution over functions through its finite marginals. By introducing a custom attention block we are able to incorporate properties of stochastic processes, such as exchangeability, directly into the NDP's architecture. We empirically show that NDPs can capture functional distributions close to the true Bayesian posterior, demonstrating that they can successfully emulate the behaviour of Gaussian processes and surpass the performance of neural processes. NDPs enable a variety of downstream tasks, including regression, implicit hyperparameter marginalisation, non-Gaussian posterior prediction and global optimisation.

翻译：基于神经网络的元学习方法在函数分布建模中具有灵活性强和推理复杂度低等理想特性。借鉴去噪扩散模型在生成建模中的成功经验，我们提出神经扩散过程（NDPs）——一种通过有限边缘分布学习从丰富函数分布中采样的创新方法。通过引入定制注意力模块，我们将随机过程的关键特性（如可交换性）直接融入NDP的架构设计中。实验证明，NDP能够捕获接近真实贝叶斯后验的函数分布，成功模拟高斯过程的行为并超越神经过程的性能。该模型支持回归、隐式超参数边缘化、非高斯后验预测及全局优化等多种下游任务。