Audio generation has long been fragmented, with speech, music, and sound effects produced by domain-specific models that fail to jointly generate coherent audio scenes from a single description. The key obstacles are insufficient fine-grained supervision for real-world mixed audio and limited acoustic representations for modeling concurrent audio components. We present Dasheng AudioGen, a unified framework for generating general mixed-audio scenes from text. Dasheng AudioGen introduces structured multi-view captions, which explicitly decouple complex acoustic scenes into complementary description views, thereby enabling fine-grained control over audio layers. Furthermore, we employ a high-dimensional unified semantic-acoustic representation as the shared latent space. It injects semantic priors that facilitate cross-modal training convergence, while its high-dimensional feature space provides sufficient capacity to disentangle and fuse concurrent audio components effectively. With these designs, a simple flow-matching DiT achieves high-quality end-to-end audio scene generation. We also establish a comprehensive evaluation pipeline for audio scene generation. Experiments demonstrate that Dasheng AudioGen achieves performance approaching real-world recordings in mixed-audio categories, while remaining competitive with specialized models in single-type generation tasks. Demos are available at https://nieeim.github.io/Dasheng-AudioGen-Web/.
翻译:音频生成长期以来呈碎片化状态,语音、音乐和音效均由领域特定模型产生,这些模型无法根据单一描述联合生成连贯的音频场景。主要障碍在于缺乏针对真实世界混合音频的细粒度监督,以及用于建模并发音频组件的声学表示有限。我们提出Dasheng AudioGen,这是一个从文本生成通用混合音频场景的统一框架。Dasheng AudioGen引入了结构化多视角描述,将复杂声学场景显式解耦为互补的描述视图,从而实现对音频层的细粒度控制。此外,我们采用高维统一语义-声学表示作为共享潜空间。它注入语义先验,促进跨模态训练收敛,同时其高维特征空间提供了足够容量,以有效解耦和融合并发音频组件。凭借这些设计,一个简单的流匹配DiT即可实现高质量端到端音频场景生成。我们还建立了一套全面的音频场景生成评估流程。实验表明,Dasheng AudioGen在混合音频类别中性能接近真实世界录音,同时在单类型生成任务中与专用模型具有竞争力。演示页面见https://nieeim.github.io/Dasheng-AudioGen-Web/。