Automatic Pronunciation Assessment (APA) plays a vital role in Computer-assisted Pronunciation Training (CAPT) when evaluating a second language (L2) learner's speaking proficiency. However, an apparent downside of most de facto methods is that they parallelize the modeling process throughout different speech granularities without accounting for the hierarchical and local contextual relationships among them. In light of this, a novel hierarchical approach is proposed in this paper for multi-aspect and multi-granular APA. Specifically, we first introduce the notion of sup-phonemes to explore more subtle semantic traits of L2 speakers. Second, a depth-wise separable convolution layer is exploited to better encapsulate the local context cues at the sub-word level. Finally, we use a score-restraint attention pooling mechanism to predict the sentence-level scores and optimize the component models with a multitask learning (MTL) framework. Extensive experiments carried out on a publicly-available benchmark dataset, viz. speechocean762, demonstrate the efficacy of our approach in relation to some cutting-edge baselines.

翻译：自动发音评估（APA）在计算机辅助发音训练（CAPT）中评估第二语言学习者口语水平时发挥着关键作用。然而，大多数现有方法的明显缺陷在于，它们在不同语音粒度上并行建模，未能考虑这些层级间的层次化局部上下文关系。为此，本文提出一种新颖的分层方法用于多维度多粒度APA。具体而言，我们首先引入超音素概念以探究L2学习者更细微的语义特征；其次，利用深度可分离卷积层更好地捕捉子词层面的局部上下文线索；最后，采用分数约束注意力池化机制预测句子级得分，并通过多任务学习框架优化各组件模型。在公开基准数据集speechocean762上进行的大量实验表明，与若干前沿基线方法相比，本方法具有显著有效性。