In recent years, there has been an increased popularity in image and speech generation using diffusion models. However, directly generating music waveforms from free-form text prompts is still under-explored. In this paper, we propose the first text-to-waveform music generation model that can receive arbitrary texts using diffusion models. We incorporate the free-form textual prompt as the condition to guide the waveform generation process of diffusion models. To solve the problem of lacking such text-music parallel data, we collect a dataset of text-music pairs from the Internet with weak supervision. Besides, we compare the effect of two prompt formats of conditioning texts (music tags and free-form texts) and prove the superior performance of our method in terms of text-music relevance. We further demonstrate that our generated music in the waveform domain outperforms previous works by a large margin in terms of diversity, quality, and text-music relevance.

翻译：近年来，利用扩散模型进行图像和语音生成日益流行。然而，直接从自由形式的文本提示生成音乐波形的研究仍处于探索阶段。本文首次提出一种基于扩散模型的文本到波形音乐生成模型，能够接收任意文本输入。我们将自由形式的文本提示作为条件，引导扩散模型的波形生成过程。为解决此类文本-音乐平行数据匮乏的问题，我们通过弱监督方式从互联网收集了包含文本-音乐配对的数据集。此外，我们比较了两种条件文本提示格式（音乐标签和自由形式文本）的效果，证明本方法在文本-音乐相关性方面具有更优性能。我们进一步证明，在波形域生成的音乐在多样性、质量及文本-音乐相关性方面显著优于先前工作。