Music timbre transfer is a challenging task that involves modifying the timbral characteristics of an audio signal while preserving its melodic structure. In this paper, we propose a novel method based on dual diffusion bridges, trained using the CocoChorales Dataset, which consists of unpaired monophonic single-instrument audio data. Each diffusion model is trained on a specific instrument with a Gaussian prior. During inference, a model is designated as the source model to map the input audio to its corresponding Gaussian prior, and another model is designated as the target model to reconstruct the target audio from this Gaussian prior, thereby facilitating timbre transfer. We compare our approach against existing unsupervised timbre transfer models such as VAEGAN and Gaussian Flow Bridges (GFB). Experimental results demonstrate that our method achieves both better Fr\'echet Audio Distance (FAD) and melody preservation, as reflected by lower pitch distances (DPD) compared to VAEGAN and GFB. Additionally, we discover that the noise level from the Gaussian prior, $\sigma$, can be adjusted to control the degree of melody preservation and amount of timbre transferred.
翻译:音乐音色转换是一项具有挑战性的任务,其目标是在保持音频信号旋律结构的同时,修改其音色特征。本文提出一种基于双扩散桥接的新方法,该方法使用CocoChorales数据集进行训练,该数据集由非配对的单音轨单乐器音频数据构成。每个扩散模型均以高斯先验为基础,针对特定乐器进行训练。在推理过程中,指定一个模型作为源模型,将输入音频映射至其对应的高斯先验;同时指定另一个模型作为目标模型,从该高斯先验重构目标音频,从而实现音色转换。我们将所提方法与现有无监督音色转换模型(如VAEGAN和高斯流桥接(GFB))进行比较。实验结果表明,相较于VAEGAN和GFB,我们的方法在弗雷歇音频距离(FAD)和旋律保持性(体现为更低的音高距离(DPD))方面均表现更优。此外,我们发现可通过调整高斯先验的噪声水平参数$\sigma$,来控制旋律保持程度与音色转换量。