Voice conversion is becoming increasingly popular, and a growing number of application scenarios require models with streaming inference capabilities. The recently proposed DualVC attempts to achieve this objective through streaming model architecture design and intra-model knowledge distillation along with hybrid predictive coding to compensate for the lack of future information. However, DualVC encounters several problems that limit its performance. First, the autoregressive decoder has error accumulation in its nature and limits the inference speed as well. Second, the causal convolution enables streaming capability but cannot sufficiently use future information within chunks. Third, the model is unable to effectively address the noise in the unvoiced segments, lowering the sound quality. In this paper, we propose DualVC 2 to address these issues. Specifically, the model backbone is migrated to a Conformer-based architecture, empowering parallel inference. Causal convolution is replaced by non-causal convolution with dynamic chunk mask to make better use of within-chunk future information. Also, quiet attention is introduced to enhance the model's noise robustness. Experiments show that DualVC 2 outperforms DualVC and other baseline systems in both subjective and objective metrics, with only 186.4 ms latency. Our audio samples are made publicly available.
翻译:语音转换技术正日益普及,越来越多的应用场景要求模型具备流式推理能力。近期提出的DualVC旨在通过流式模型架构设计、结合混合预测编码的模型内知识蒸馏技术弥补未来信息缺失,从而实现这一目标。然而,DualVC存在若干限制性能的问题:首先,自回归解码器固有的误差累积特性不仅影响音质,还限制了推理速度;其次,因果卷积虽支持流式处理,但无法充分利用分块内的未来信息;第三,模型难以有效处理非语音段噪声,导致音质下降。本文提出DualVC 2以解决上述问题。具体而言,模型主干迁移至基于Conformer的架构以实现并行推理;采用非因果卷积配合动态分块掩码替代因果卷积,更充分地利用分块内未来信息;同时引入静音注意力机制增强模型噪声鲁棒性。实验表明,DualVC 2在仅186.4毫秒延迟条件下,主观与客观指标均优于DualVC及其他基线系统。我们的音频样本已公开发布。