This paper introduces Bayesian Flow Networks (BFNs), a new class of generative model in which the parameters of a set of independent distributions are modified with Bayesian inference in the light of noisy data samples, then passed as input to a neural network that outputs a second, interdependent distribution. Starting from a simple prior and iteratively updating the two distributions yields a generative procedure similar to the reverse process of diffusion models; however it is conceptually simpler in that no forward process is required. Discrete and continuous-time loss functions are derived for continuous, discretised and discrete data, along with sample generation procedures. Notably, the network inputs for discrete data lie on the probability simplex, and are therefore natively differentiable, paving the way for gradient-based sample guidance and few-step generation in discrete domains such as language modelling. The loss function directly optimises data compression and places no restrictions on the network architecture. In our experiments BFNs achieve competitive log-likelihoods for image modelling on dynamically binarized MNIST and CIFAR-10, and outperform all known discrete diffusion models on the text8 character-level language modelling task.

翻译：本文提出贝叶斯流网络（Bayesian Flow Networks, BFNs），这是一类新型生成模型。在该模型中，一组独立分布的参数通过贝叶斯推理依据含噪数据样本进行修正，随后作为输入传递给神经网络，输出第二组相互依赖的分布。从简单先验出发，迭代更新这两个分布可生成类似于扩散模型逆过程的生成过程；但其概念更简洁，无需正向过程。针对连续型、离散化及离散型数据，本文推导了离散时间与连续时间损失函数，并给出了样本生成流程。值得注意的是，离散型数据的网络输入位于概率单纯形上，因此天然可微，为基于梯度的样本引导及离散领域（如语言建模）中的少步生成铺平了道路。该损失函数直接优化数据压缩，且对网络架构无任何限制。实验中，BFNs在动态二值化MNIST和CIFAR-10图像建模任务中取得了具有竞争力的对数似然值，并在text8字符级语言建模任务中超越了所有已知的离散扩散模型。