Diffusion models have achieved state-of-the-art performance, demonstrating remarkable generalisation capabilities across diverse domains. However, the mechanisms underpinning these strong capabilities remain only partially understood. A leading conjecture, based on the manifold hypothesis, attributes this success to their ability to adapt to low-dimensional geometric structure within the data. This work provides evidence for this conjecture, focusing on how such phenomena could result from the formulation of the learning problem through score matching. We inspect the role of implicit regularisation by investigating the effect of smoothing minimisers of the empirical score matching objective. Our theoretical and empirical results confirm that smoothing the score function -- or equivalently, smoothing in the log-density domain -- produces smoothing tangential to the data manifold. In addition, we show that the manifold along which the diffusion model generalises can be controlled by choosing an appropriate smoothing.

翻译：扩散模型已取得最先进的性能，在多个领域展现出卓越的泛化能力。然而，支撑这些强大能力的机制仍未得到充分理解。基于流形假设的主流推测认为，其成功归因于模型对数据内在低维几何结构的自适应能力。本研究为该推测提供了证据，重点探讨了此类现象如何通过分数匹配的学习问题表述而产生。我们通过研究经验分数匹配目标最小化子的平滑效应，深入分析了隐式正则化的作用。理论与实证结果均证实：对分数函数进行平滑（等价于在对数密度域进行平滑）会产生沿数据流形切向的平滑效果。此外，我们还证明通过选择适当的平滑方式，可以控制扩散模型进行泛化的流形结构。