Diffusion models have gained prominence in the image domain for their capabilities in data generation and transformation, achieving state-of-the-art performance in various tasks in both image and audio domains. In the rapidly evolving field of audio-based machine learning, safeguarding model integrity and establishing data copyright are of paramount importance. This paper presents the first watermarking technique applied to audio diffusion models trained on mel-spectrograms. This offers a novel approach to the aforementioned challenges. Our model excels not only in benign audio generation, but also incorporates an invisible watermarking trigger mechanism for model verification. This watermark trigger serves as a protective layer, enabling the identification of model ownership and ensuring its integrity. Through extensive experiments, we demonstrate that invisible watermark triggers can effectively protect against unauthorized modifications while maintaining high utility in benign audio generation tasks.

翻译：扩散模型因其在数据生成与转换方面的卓越能力，已在图像领域获得广泛关注，并在图像及音频领域的多项任务中达到先进水平。在基于音频的机器学习这一快速发展的领域中，保护模型完整性及确立数据版权至关重要。本文首次提出一种应用于基于梅尔频谱图训练的音频扩散模型的水印技术，为解决上述挑战提供了新方法。我们的模型不仅能在良性音频生成中表现出色，还集成了用于模型验证的不可见水印触发机制。该水印触发机制作为保护层，能够识别模型所有权并确保其完整性。通过大量实验证明，不可见水印触发机制可有效防止未经授权的篡改，同时保持良性音频生成任务的高实用性。