Articulatory recordings track the positions and motion of different articulators along the vocal tract and are widely used to study speech production and to develop speech technologies such as articulatory based speech synthesizers and speech inversion systems. The University of Wisconsin X-Ray microbeam (XRMB) dataset is one of various datasets that provide articulatory recordings synced with audio recordings. The XRMB articulatory recordings employ pellets placed on a number of articulators which can be tracked by the microbeam. However, a significant portion of the articulatory recordings are mistracked, and have been so far unsuable. In this work, we present a deep learning based approach using Masked Autoencoders to accurately reconstruct the mistracked articulatory recordings for 41 out of 47 speakers of the XRMB dataset. Our model is able to reconstruct articulatory trajectories that closely match ground truth, even when three out of eight articulators are mistracked, and retrieve 3.28 out of 3.4 hours of previously unusable recordings.

翻译：发音记录追踪声道中不同发音器官的位置和运动，广泛应用于语音产生研究以及基于发音的语音合成器和语音反演系统等语音技术的发展。威斯康星大学X射线微束（XRMB）数据集是提供与音频记录同步的发音记录的多个数据集之一。XRMB发音记录通过在多个发音器官上放置可被微束追踪的金属片来获取。然而，相当一部分发音记录存在追踪错误，迄今为止无法使用。在这项工作中，我们提出了一种基于深度学习的方法，利用掩码自编码器精确重建XRMB数据集中47名说话者中41名的错误追踪发音记录。我们的模型能够重建与真实情况高度吻合的发音轨迹，即使在八个发音器官中有三个追踪错误的情况下，也能恢复3.4小时中3.28小时先前不可用的记录。