Music timbre transfer is a challenging task that involves modifying the timbral characteristics of an audio signal while preserving its melodic structure. In this paper, we propose a novel method based on dual diffusion bridges, trained using the CocoChorales Dataset, which consists of unpaired monophonic single-instrument audio data. Each diffusion model is trained on a specific instrument with a Gaussian prior. During inference, a model is designated as the source model to map the input audio to its corresponding Gaussian prior, and another model is designated as the target model to reconstruct the target audio from this Gaussian prior, thereby facilitating timbre transfer. We compare our approach against existing unsupervised timbre transfer models such as VAEGAN and Gaussian Flow Bridges (GFB). Experimental results demonstrate that our method achieves both better Fr\'echet Audio Distance (FAD) and melody preservation, as reflected by lower pitch distances (DPD) compared to VAEGAN and GFB. Additionally, we discover that the noise level from the Gaussian prior, $\sigma$, can be adjusted to control the degree of melody preservation and amount of timbre transferred.
翻译:音乐音色转换是一项具有挑战性的任务,它要求在保持音频信号旋律结构的同时,修改其音色特征。本文提出了一种基于双扩散桥的新方法,该方法使用CocoChorales数据集进行训练,该数据集由非配对的单声道单乐器音频数据组成。每个扩散模型都在特定乐器上以高斯先验进行训练。在推理过程中,指定一个模型作为源模型,将输入音频映射到其对应的高斯先验;指定另一个模型作为目标模型,从该高斯先验重建目标音频,从而实现音色转换。我们将我们的方法与现有的无监督音色转换模型(如VAEGAN和高斯流桥GFB)进行了比较。实验结果表明,与VAEGAN和GFB相比,我们的方法在Fréchet音频距离(FAD)和旋律保持方面都表现更优,这体现在更低的音高距离(DPD)上。此外,我们发现可以通过调整高斯先验的噪声水平$\sigma$来控制旋律保持的程度和音色转换的量。