We introduce Noise2Music, where a series of diffusion models is trained to generate high-quality 30-second music clips from text prompts. Two types of diffusion models, a generator model, which generates an intermediate representation conditioned on text, and a cascader model, which generates high-fidelity audio conditioned on the intermediate representation and possibly the text, are trained and utilized in succession to generate high-fidelity music. We explore two options for the intermediate representation, one using a spectrogram and the other using audio with lower fidelity. We find that the generated audio is not only able to faithfully reflect key elements of the text prompt such as genre, tempo, instruments, mood, and era, but goes beyond to ground fine-grained semantics of the prompt. Pretrained large language models play a key role in this story -- they are used to generate paired text for the audio of the training set and to extract embeddings of the text prompts ingested by the diffusion models. Generated examples: https://google-research.github.io/noise2music

翻译：我们提出Noise2Music，通过训练一系列扩散模型，从文本提示生成高质量30秒音乐片段。两种扩散模型——生成器模型（基于文本条件生成中间表示）与级联器模型（基于中间表示及文本条件生成高保真音频）——被依次训练并用于生成高保真音乐。我们探索了两种中间表示方案：一种采用频谱图，另一种采用低保真音频。研究发现，生成音频不仅能忠实反映文本提示的关键要素（如流派、速度、乐器、情绪和时代），还能深入解析文本提示的细粒度语义。预训练大语言模型在此过程中发挥关键作用——它们被用于为训练集音频生成配对文本，并提取扩散模型所需的文本提示嵌入。生成示例：https://google-research.github.io/noise2music