The landscape of high-performance image generation models is currently shifting from the inefficient multi-step ones to the efficient few-step counterparts (e.g, Z-Image-Turbo and FLUX.2-klein). However, these models present significant challenges for directly continuous supervised fine-tuning. For example, applying the commonly used fine-tuning technique would compromises their inherent few-step inference capability. To address this, we propose D-OPSD, a novel training paradigm for step-distilled diffusion models that enables on-policy learning during supervised fine-tuning. We first find that the modern diffusion model where the LLM/VLM serves as the encoder can inherit its encoder's in-context capabilities. This enables us to make the training as an on-policy self-distillation process. Specifically, during training, we make the model acts as both the teacher and the student with different contexts, where the student is conditioned only on the text feature, while the teacher is conditioned on the multimodal feature of both the text prompt and the target image. Training minimizes the two predicted distributions over the student's own roll-outs. By optimized on the model's own trajectory and under it's own supervision, D-OPSD enables the model to learn new concept, style, etc. without sacrificing the original few-step capacity.

翻译：高性能图像生成模型的格局目前正从低效的多步模型向高效的少步模型（如Z-Image-Turbo和FLUX.2-klein）转变。然而，这些模型在直接进行连续监督微调时面临显著挑战。例如，应用常用的微调技术会损害其固有的少步推理能力。为解决这一问题，我们提出D-OPSD，一种面向步蒸馏扩散模型的新型训练范式，能够在监督微调过程中实现基于策略的学习。我们首先发现，以LLM/VLM为编码器的现代扩散模型能够继承其编码器的上下文能力。这使我们能够将训练过程构建为基于策略的自蒸馏过程。具体而言，在训练阶段，我们让模型在不同上下文中同时扮演教师和学生角色：学生仅以文本特征为条件，而教师则以文本提示和目标图像的多模态特征为条件。训练过程中，模型通过最小化学生自身轨迹上的两个预测分布进行优化。通过基于模型自身轨迹并在其自身监督下进行优化，D-OPSD使模型能够在不牺牲原始少步能力的前提下学习新概念、风格等。