Diffusion models have demonstrated remarkable performance in generating high-dimensional samples across domains such as vision, language, and the sciences. Although continuous-state diffusion models have been extensively studied both empirically and theoretically, discrete-state diffusion models, essential for applications involving text, sequences, and combinatorial structures, remain significantly less understood from a theoretical standpoint. In particular, all existing analyses of discrete-state models assume score estimation error bounds without studying sample complexity results. In this work, we present a principled theoretical framework for discrete-state diffusion, providing the first sample complexity bound of $\widetilde{\mathcal{O}}(ε^{-2})$. Our structured decomposition of the score estimation error into statistical, approximation, optimization, and clipping components offers critical insights into how discrete-state models can be trained efficiently. This analysis addresses a fundamental gap in the literature and establishes the theoretical tractability and practical relevance of discrete-state diffusion models.

翻译：扩散模型在生成高维样本方面展现出卓越性能，其应用涵盖视觉、语言及科学计算等多个领域。尽管连续状态扩散模型已在实证与理论层面得到广泛研究，但针对涉及文本、序列及组合结构应用至关重要的离散状态扩散模型，其理论认知仍存在显著不足。特别值得注意的是，现有对离散状态模型的所有分析均假设得分估计误差界成立，而未深入探究样本复杂度的理论结果。本研究提出一个基于原理的离散状态扩散理论框架，首次给出$\widetilde{\mathcal{O}}(ε^{-2})$的样本复杂度上界。通过将得分估计误差结构化分解为统计误差、近似误差、优化误差与截断误差四个组成部分，本研究揭示了离散状态模型高效训练的内在机制。该分析填补了现有文献的关键理论空白，确立了离散状态扩散模型的理论可处理性与实践相关性。