This work presents self-rewarding sequential Monte Carlo (SMC), an inference-time scaling algorithm enabling effective sampling of masked diffusion language models (MDLMs). Our algorithm stems from the observation that most existing MDLMs rely on a confidence-based sampling strategy, where only tokens with the highest prediction confidence are preserved at each step. This restricts the generation to a noise-sensitive, greedy decoding paradigm, resulting in an inevitable collapse in the diversity of possible paths. We address this problem by launching multiple interacting diffusion processes in parallel, referred to as particles, for trajectory exploration. Importantly, we introduce the trajectory-level confidence as a self-rewarding signal for assigning particle importance weights. During sampling, particles are iteratively weighted and resampled to systematically steer generation towards globally confident, high-quality samples. Our self-rewarding SMC is verified on various masked diffusion language models and benchmarks, achieving significant improvement without extra training or reward guidance, while effectively converting parallel inference capacity into improved sampling quality. Our code is available at https://github.com/Algolzw/self-rewarding-smc.

翻译：本文提出了一种自奖励序列蒙特卡洛（SMC）方法，这是一种推理时扩展算法，能够有效采样掩码扩散语言模型（MDLM）。该算法的提出源于我们观察到现有大多数MDLM依赖于基于置信度的采样策略，即在每一步仅保留预测置信度最高的词元。这种做法将生成过程限制在一种对噪声敏感、贪婪的解码范式内，导致可能路径的多样性不可避免地坍缩。为解决该问题，我们并行启动多个相互作用的扩散过程（称为粒子）以进行轨迹探索。重要的是，我们引入了轨迹级置信度作为自奖励信号，用于分配粒子重要性权重。在采样过程中，粒子被迭代地加权和重采样，以系统地将生成过程导向全局置信度高、质量优良的样本。我们在多种掩码扩散语言模型和基准测试上验证了自奖励SMC的有效性，该方法在不进行额外训练或奖励引导的情况下实现了显著提升，同时有效地将并行推理能力转化为改进的采样质量。我们的代码公开于 https://github.com/Algolzw/self-rewarding-smc。