Recent progress in diffusion-based audio generation and restoration has substantially improved performance across heterogeneous conditioning regimes, including text-conditioned audio generation and audio-conditioned super-resolution. However, training audio diffusion models remains computationally expensive, and most existing pipelines still rely on static optimization recipes that treat the relative importance of training signals as fixed throughout learning. In this work, we argue that a major source of inefficiency lies in the evolving balance between semantic acquisition and generation-oriented refinement. Early training places stronger emphasis on acquiring condition-aligned semantic structure and coarse global organization, whereas later training increasingly emphasizes temporal consistency, perceptual fidelity, and fine-detail refinement. To characterize this evolving balance, we introduce a progress-based regime variable derived from the training-time slope of an SSL-space discrepancy, which measures semantic progress during training. Based on this signal, we develop three complementary stage-aware mechanisms: decayed SSL guidance for early semantic bootstrapping, self-adaptive timestep sampling driven by the regime variable, and structure-aware regularization activated from convergent grouped organization in parameter space. We evaluate these mechanisms on text-conditioned audio generation and audio-conditioned super-resolution. Across both settings, the proposed stage-aware strategies improve convergence behavior and yield gains on the primary generation and spectral reconstruction metrics over standard static baselines. These results support the view that efficient audio diffusion training can benefit from treating external guidance, internal organization, and optimization emphasis as stage-dependent components rather than fixed ingredients.

翻译：近期，基于扩散的音频生成与修复技术在多模态条件框架下取得了显著性能提升，涵盖文本条件音频生成与音频条件超分辨率重建。然而，训练音频扩散模型仍存在计算成本高昂的瓶颈，且现有训练管线多采用静态优化策略，将训练信号的重要程度设定为恒定值。本文指出，学习效率低下的主要根源在于语义获取与生成优化之间动态平衡的演变过程：早期训练阶段更注重条件对齐的语义结构与全局粗粒度组织，而后期的训练重心逐渐转向时间一致性、感知保真度与细节精细优化。为刻画这种演化平衡，我们引入基于自监督学习（SSL）空间差异的训练时间梯度的进度变量，用以度量训练过程中的语义进展。基于该信号，我们开发了三种互补的阶段感知机制：用于早期语义引导的衰减式SSL引导、由进度变量驱动的自适应时间步采样策略，以及基于参数空间收敛簇群结构激活的结构感知正则化。我们在文本条件音频生成与音频条件超分辨率两个任务上验证了这些机制的有效性。实验结果表明，相较于标准静态基线，所提出的阶段感知策略显著改善了收敛行为，并在主要生成指标与频谱重建指标上均取得性能提升。这些成果证实，将外部引导、内部组织架构及优化重点视为阶段依赖性组件而非固定要素，是提升音频扩散训练效率的关键路径。