Editing signals using large pre-trained models, in a zero-shot manner, has recently seen rapid advancements in the image domain. However, this wave has yet to reach the audio domain. In this paper, we explore two zero-shot editing techniques for audio signals, which use DDPM inversion on pre-trained diffusion models. The first, adopted from the image domain, allows text-based editing. The second, is a novel approach for discovering semantically meaningful editing directions without supervision. When applied to music signals, this method exposes a range of musically interesting modifications, from controlling the participation of specific instruments to improvisations on the melody. Samples and code can be found on our examples page in https://hilamanor.github.io/AudioEditing/ .
翻译:最近,利用大规模预训练模型以零样本方式编辑信号在图像领域取得了快速进展,然而这一浪潮尚未波及音频领域。本文探索了两种用于音频信号的零样本编辑技术,这两种技术均基于预训练扩散模型上的DDPM逆映射。第一种技术源自图像领域,支持基于文本的编辑;第二种技术则是一种无监督发现语义上有意义的编辑方向的新方法。当应用于音乐信号时,该方法揭示了从控制特定乐器的参与度到旋律即兴创作等一系列音乐上有趣的修改。示例与代码可访问我们的演示页面https://hilamanor.github.io/AudioEditing/。