Combining the merits of both denoising diffusion probabilistic models and gradient boosting, the diffusion boosting paradigm is introduced for tackling supervised learning problems. We develop Diffusion Boosted Trees (DBT), which can be viewed as both a new denoising diffusion generative model parameterized by decision trees (one single tree for each diffusion timestep), and a new boosting algorithm that combines the weak learners into a strong learner of conditional distributions without making explicit parametric assumptions on their density forms. We demonstrate through experiments the advantages of DBT over deep neural network-based diffusion models as well as the competence of DBT on real-world regression tasks, and present a business application (fraud detection) of DBT for classification on tabular data with the ability of learning to defer.

翻译：结合去噪扩散概率模型与梯度提升的优势，本文提出扩散提升范式以解决监督学习问题。我们开发了扩散提升树（DBT），该模型既可视为一种由决策树参数化的新型去噪扩散生成模型（每个扩散时间步对应单棵树），也可视为一种新的提升算法——该算法将弱学习器组合成条件分布的强学习器，且无需对其密度形式进行显式参数化假设。实验证明DBT相较于基于深度神经网络的扩散模型具有优势，并在现实世界回归任务中展现出竞争力。本文还展示了DBT在表格数据分类中的商业应用（欺诈检测），该模型具备学习延迟决策的能力。