Vocal tract articulation is a natural, grounded control space of speech production. The spatiotemporal coordination of articulators combined with the vocal source shapes intelligible speech sounds to enable effective spoken communication. Based on this physiological grounding of speech, we propose a new framework of neural encoding-decoding of speech -- Speech Articulatory Coding (SPARC). SPARC comprises an articulatory analysis model that infers articulatory features from speech audio, and an articulatory synthesis model that synthesizes speech audio from articulatory features. The articulatory features are kinematic traces of vocal tract articulators and source features, which are intuitively interpretable and controllable, being the actual physical interface of speech production. An additional speaker identity encoder is jointly trained with the articulatory synthesizer to inform the voice texture of individual speakers. By training on large-scale speech data, we achieve a fully intelligible, high-quality articulatory synthesizer that generalizes to unseen speakers. Furthermore, the speaker embedding is effectively disentangled from articulations, which enables accent-perserving zero-shot voice conversion. To the best of our knowledge, this is the first demonstration of universal, high-performance articulatory inference and synthesis, suggesting the proposed framework as a powerful coding system of speech.

翻译：声道发音是语音产生的一种自然、基础的控制空间。发音器官的时空协调与声源共同塑造可理解的语音，从而实现有效的口语交流。基于语音的这一生理学基础，我们提出了一种新的神经语音编码-解码框架——语音发音编码（SPARC）。SPARC包含一个从语音音频推断发音特征的发音分析模型，以及一个从发音特征合成语音音频的发音合成模型。发音特征包括声道发音器官的运动轨迹特征和声源特征，这些特征直观可解释且可控制，是语音产生的实际物理接口。一个额外的说话人身份编码器与发音合成器联合训练，以传递个体说话人的音色特征。通过在大规模语音数据上训练，我们实现了一个完全可理解、高质量的发音合成器，能够泛化到未见过的说话人。此外，说话人嵌入与发音特征有效解耦，这使得保持口音的零样本语音转换成为可能。据我们所知，这是首次实现通用、高性能的发音推断与合成，表明所提出的框架是一个强大的语音编码系统。