Diffusion models perform remarkably well on high-dimensional data such as images, often using only a modest number of reverse-time steps. Despite this practical success, existing convergence theory does not fully explain why such samplers remain efficient in high dimensions. Many prior KL guarantees bound the discretization error in terms of the ambient dimension, while other improved results replace this dependence using intrinsic-dimensional or geometric structure assumptions. In this work, we develop an alternative information-theoretic perspective on diffusion sampler convergence. We prove that, for Gaussian mixture targets, the discretization error is controlled by the Shannon entropy of the latent mixture component rather than by the ambient dimension. Consequently, the leading step complexity scales linearly with latent entropy and depends only logarithmically on the second moment of the data. Our analysis also extends to discrete target distributions, where the relevant complexity is the entropy of the target rather than the dimension of the embedding space. These results suggest that diffusion sampling can remain efficient in high-dimensional spaces when the data distribution admits a compact latent representation, as is widely believed to be the case for natural images.

翻译：扩散模型在高维数据（如图像）上表现出色，通常仅需使用适中数量的反向时间步长。尽管取得了实际成功，但现有的收敛理论未能完全解释为何这类采样器在高维空间中仍保持高效。许多先前的KL散度保证将离散化误差与环境维度相关联，而其他改进结果则通过内在维度或几何结构假设替代了这一依赖关系。在本工作中，我们发展了一种关于扩散采样器收敛的替代性信息论视角。我们证明，对于高斯混合目标，离散化误差由潜在混合成分的香农熵控制，而非环境维度。因此，主要步骤复杂度随潜在熵线性增长，且仅依赖于数据二阶矩的对数尺度。我们的分析还扩展到离散目标分布，其中相关复杂度是目标的熵而非嵌入空间的维度。这些结果表明，当数据分布具有紧凑的潜在表示时（正如自然图像被广泛认为的那样），扩散采样可在高维空间中保持高效。