Perceptual modification of voice is an elusive goal. While non-experts can modify an image or sentence perceptually with available tools, it is not clear how to similarly modify speech along perceptual axes. Voice conversion does make it possible to convert one voice to another, but these modifications are handled by black box models, and the specifics of what perceptual qualities to modify and how to modify them are unclear. Towards allowing greater perceptual control over voice, we introduce PerMod, a conditional latent diffusion model that takes in an input voice and a perceptual qualities vector, and produces a voice with the matching perceptual qualities. Unlike prior work, PerMod generates a new voice corresponding to specific perceptual modifications. Evaluating perceptual quality vectors with RMSE from both human and predicted labels, we demonstrate that PerMod produces voices with the desired perceptual qualities for typical voices, but performs poorly on atypical voices.

翻译：对语音进行感知层面的修改是一个难以实现的目标。尽管非专家用户可以使用现有工具对图像或句子进行感知层面的修改，但如何类似地沿感知维度修改语音尚不明确。语音转换虽然能将一种语音转换为另一种，但这些修改由黑箱模型处理，具体要修改哪些感知特性以及如何修改仍不清楚。为赋予用户对语音的更强的感知控制能力，我们提出PerMod——一种条件潜扩散模型，它接收输入语音和感知特性向量，并生成匹配这些感知特性的语音。与先前工作不同，PerMod能够生成对应特定感知修改的新语音。通过使用人类标注和预测标签对感知特性向量计算均方根误差（RMSE），我们证明PerMod能对典型语音生成具有预期感知特性的语音，但在非典型语音上表现欠佳。