Music timbre transfer is a challenging task that involves modifying the timbral characteristics of an audio signal while preserving its melodic structure. In this paper, we propose a novel method based on dual diffusion bridges, trained using the CocoChorales Dataset, which consists of unpaired monophonic single-instrument audio data. Each diffusion model is trained on a specific instrument with a Gaussian prior. During inference, a model is designated as the source model to map the input audio to its corresponding Gaussian prior, and another model is designated as the target model to reconstruct the target audio from this Gaussian prior, thereby facilitating timbre transfer. We compare our approach against existing unsupervised timbre transfer models such as VAEGAN and Gaussian Flow Bridges (GFB). Experimental results demonstrate that our method achieves both better Fr\'echet Audio Distance (FAD) and melody preservation, as reflected by lower pitch distances (DPD) compared to VAEGAN and GFB. Additionally, we discover that the noise level from the Gaussian prior, $\sigma$, can be adjusted to control the degree of melody preservation and amount of timbre transferred.
翻译:音乐音色转换是一项具有挑战性的任务,它要求在保持音频信号旋律结构的同时,修改其音色特征。本文提出了一种基于双扩散桥的新方法,该方法使用CocoChorales数据集进行训练,该数据集包含未配对的单音轨单乐器音频数据。每个扩散模型均以高斯先验为条件,针对特定乐器进行训练。在推理过程中,指定一个模型作为源模型,将输入音频映射到其对应的高斯先验;指定另一个模型作为目标模型,从该高斯先验重构目标音频,从而实现音色转换。我们将所提方法与现有的无监督音色转换模型(如VAEGAN和高斯流桥GFB)进行了比较。实验结果表明,与VAEGAN和GFB相比,我们的方法在取得更优弗雷歇音频距离(FAD)的同时,也实现了更好的旋律保持性,这体现在更低的音高距离(DPD)上。此外,我们发现可以通过调整高斯先验的噪声水平$\sigma$,来控制旋律保持的程度与音色转换的量。