Diffusion models have gained prominence in the image domain for their capabilities in data generation and transformation, achieving state-of-the-art performance in various tasks in both image and audio domains. In the rapidly evolving field of audio-based machine learning, safeguarding model integrity and establishing data copyright are of paramount importance. This paper presents the first watermarking technique applied to audio diffusion models trained on mel-spectrograms. This offers a novel approach to the aforementioned challenges. Our model excels not only in benign audio generation, but also incorporates an invisible watermarking trigger mechanism for model verification. This watermark trigger serves as a protective layer, enabling the identification of model ownership and ensuring its integrity. Through extensive experiments, we demonstrate that invisible watermark triggers can effectively protect against unauthorized modifications while maintaining high utility in benign audio generation tasks.

翻译：扩散模型因其在数据生成与转换方面的强大能力而在图像领域崭露头角，并在图像和音频领域的多项任务中达到了最先进水平。在快速发展的基于音频的机器学习领域中，保护模型完整性和确立数据版权至关重要。本文首次提出了一种应用于基于梅尔频谱图训练的音频扩散模型的水印技术，为上述挑战提供了一种新颖的解决方案。我们的模型不仅在良性音频生成中表现出色，还集成了不可听见的水印触发机制，用于模型验证。该水印触发机制作为一种保护层，使得模型所有权的识别成为可能，同时保障了模型的完整性。通过大量实验，我们证明不可听见的水印触发器能够有效防御未授权篡改，同时保持良性音频生成任务中的高实用性。