Recent advances in diffusion-based generative models have enabled high-quality text-to-audio synthesis, but fine-grained acoustic control remains a significant challenge in open-source research. We present Audio Palette, a diffusion transformer (DiT) based model that extends the Stable Audio Open architecture to address this "control gap" in controllable audio generation. Unlike prior approaches that rely solely on semantic conditioning, Audio Palette introduces four time-varying control signals: loudness, pitch, spectral centroid, and timbre, for precise and interpretable manipulation of acoustic features. The model is efficiently adapted for the nuanced domain of Foley synthesis using Low-Rank Adaptation (LoRA) on a curated subset of AudioSet, requiring only 0.85 percent of the original parameters to be trained. Experiments demonstrate that Audio Palette achieves fine-grained, interpretable control of sound attributes. Crucially, it accomplishes this novel controllability while maintaining high audio quality and strong semantic alignment to text prompts, with performance on standard metrics such as Frechet Audio Distance (FAD) and LAION-CLAP scores remaining comparable to the original baseline model. We provide a scalable, modular pipeline for audio research, emphasizing sequence-based conditioning, memory efficiency, and a three-scale classifier-free guidance mechanism for nuanced inference-time control. This work establishes a robust foundation for controllable sound design and performative audio synthesis in open-source settings, enabling a more artist-centric workflow.

翻译：近年来，基于扩散的生成模型在文本到音频合成方面取得了高质量成果，但细粒度的声学控制在开源研究中仍是一个重大挑战。我们提出了音频调色板，这是一种基于扩散Transformer的模型，它扩展了Stable Audio Open架构，以解决可控音频生成中的“控制鸿沟”。与以往仅依赖语义条件的方法不同，音频调色板引入了四种时变控制信号：响度、音高、频谱质心和音色，以实现对声学特征的精确且可解释的操控。该模型通过在AudioSet的精选子集上使用低秩适应技术，高效地适应了拟音合成这一精细领域，仅需训练原始参数的0.85%。实验表明，音频调色板实现了对声音属性的细粒度、可解释控制。至关重要的是，它在实现这种新颖可控性的同时，保持了高音频质量以及与文本提示的强语义对齐，在弗雷歇音频距离和LAION-CLAP评分等标准指标上的性能与原始基线模型保持相当。我们提供了一个可扩展、模块化的音频研究流程，强调基于序列的条件化、内存效率以及用于精细推理时控制的三尺度无分类器引导机制。这项工作为开源环境下的可控声音设计和表演性音频合成奠定了坚实基础，实现了更加以艺术家为中心的工作流程。