Automatic Pronunciation Assessment (APA) plays a vital role in Computer-assisted Pronunciation Training (CAPT) when evaluating a second language (L2) learner's speaking proficiency. However, an apparent downside of most de facto methods is that they parallelize the modeling process throughout different speech granularities without accounting for the hierarchical and local contextual relationships among them. In light of this, a novel hierarchical approach is proposed in this paper for multi-aspect and multi-granular APA. Specifically, we first introduce the notion of sup-phonemes to explore more subtle semantic traits of L2 speakers. Second, a depth-wise separable convolution layer is exploited to better encapsulate the local context cues at the sub-word level. Finally, we use a score-restraint attention pooling mechanism to predict the sentence-level scores and optimize the component models with a multitask learning (MTL) framework. Extensive experiments carried out on a publicly-available benchmark dataset, viz. speechocean762, demonstrate the efficacy of our approach in relation to some cutting-edge baselines.

翻译：自动发音评估（APA）在计算机辅助发音训练（CAPT）中评估第二语言（L2）学习者的口语水平时发挥着关键作用。然而，大多数实际方法的一个明显缺点是，它们在不同语音粒度上并行化建模过程，而未考虑这些粒度之间的层次性和局部上下文关系。鉴于此，本文提出一种新颖的层次化方法用于多维度、多粒度的APA。具体而言，我们首先引入超音位（sup-phonemes）概念以探索L2学习者更细微的语义特征。其次，利用深度可分离卷积层更好地封装子词级别的局部上下文线索。最后，采用分数约束注意力池化机制预测句子级别得分，并通过多任务学习（MTL）框架优化组件模型。在公开基准数据集speechocean762上进行的大量实验表明，我们的方法相较于若干前沿基线具有显著有效性。