We introduce beta diffusion, a novel generative modeling method that integrates demasking and denoising to generate data within bounded ranges. Using scaled and shifted beta distributions, beta diffusion utilizes multiplicative transitions over time to create both forward and reverse diffusion processes, maintaining beta distributions in both the forward marginals and the reverse conditionals, given the data at any point in time. Unlike traditional diffusion-based generative models relying on additive Gaussian noise and reweighted evidence lower bounds (ELBOs), beta diffusion is multiplicative and optimized with KL-divergence upper bounds (KLUBs) derived from the convexity of the KL divergence. We demonstrate that the proposed KLUBs are more effective for optimizing beta diffusion compared to negative ELBOs, which can also be derived as the KLUBs of the same KL divergence with its two arguments swapped. The loss function of beta diffusion, expressed in terms of Bregman divergence, further supports the efficacy of KLUBs for optimization. Experimental results on both synthetic data and natural images demonstrate the unique capabilities of beta diffusion in generative modeling of range-bounded data and validate the effectiveness of KLUBs in optimizing diffusion models, thereby making them valuable additions to the family of diffusion-based generative models and the optimization techniques used to train them.

翻译：我们提出Beta扩散，一种新颖的生成建模方法，它整合了去遮蔽与去噪过程，以在有限范围内生成数据。通过使用缩放和平移的Beta分布，Beta扩散利用随时间变化的乘性过渡构建前向与反向扩散过程，在给定任意时刻数据的条件下，保持前向边缘分布和反向条件分布均为Beta分布。不同于依赖加性高斯噪声与重加权证据下界（ELBO）的传统扩散生成模型，Beta扩散是乘性的，并通过由KL散度凸性导出的KL散度上界（KLUB）进行优化。我们证明，与负ELBO相比，所提出的KLUB能更有效地优化Beta扩散；而负ELBO本身也可视为同一KL散度交换两个参数后得到的KLUB。Beta扩散的损失函数以Bregman散度表示，进一步支持了KLUB在优化中的有效性。在合成数据和自然图像上的实验结果展示了Beta扩散在有限范围数据生成建模中的独特能力，验证了KLUB在优化扩散模型中的有效性，从而使其成为扩散生成模型家族及其训练优化技术的重要补充。