Acoustic-to-articulatory inversion (AAI) aims to estimate the parameters of articulators from speech audio. There are two common challenges in AAI, which are the limited data and the unsatisfactory performance in speaker independent scenario. Most current works focus on extracting features directly from speech and ignoring the importance of phoneme information which may limit the performance of AAI. To this end, we propose a novel network called SPN that uses two different streams to carry out the AAI task. Firstly, to improve the performance of speaker-independent experiment, we propose a new phoneme stream network to estimate the articulatory parameters as the phoneme features. To the best of our knowledge, this is the first work that extracts the speaker-independent features from phonemes to improve the performance of AAI. Secondly, in order to better represent the speech information, we train a speech stream network to combine the local features and the global features. Compared with state-of-the-art (SOTA), the proposed method reduces 0.18mm on RMSE and increases 6.0% on Pearson correlation coefficient in the speaker-independent experiment. The code has been released at https://github.com/liujinyu123/AAINetwork-SPN.

翻译：摘要：声学-发音反演旨在从语音音频中估计发音器官的参数。说话人无关场景中，数据受限和性能不佳是AAI面临的两个常见挑战。当前多数工作直接提取语音特征而忽略了音素信息的重要性，这限制了AAI的性能。为此，我们提出一种名为SPN的新型网络，采用双流架构执行AAI任务。首先，为提升说话人无关实验的性能，我们提出一种新的音素流网络，将发音参数作为音素特征进行估计。据我们所知，这是首个利用音素提取说话人无关特征以提升AAI性能的工作。其次，为更好表征语音信息，我们训练了一个语音流网络来融合局部特征与全局特征。与当前最先进方法相比，所提方法在说话人无关实验中使均方根误差降低0.18mm，皮尔逊相关系数提升6.0%。代码已开源至https://github.com/liujinyu123/AAINetwork-SPN。