Automatic Pronunciation Assessment (APA) plays a vital role in Computer-assisted Pronunciation Training (CAPT) when evaluating a second language (L2) learner's speaking proficiency. However, an apparent downside of most de facto methods is that they parallelize the modeling process throughout different speech granularities without accounting for the hierarchical and local contextual relationships among them. In light of this, a novel hierarchical approach is proposed in this paper for multi-aspect and multi-granular APA. Specifically, we first introduce the notion of sup-phonemes to explore more subtle semantic traits of L2 speakers. Second, a depth-wise separable convolution layer is exploited to better encapsulate the local context cues at the sub-word level. Finally, we use a score-restraint attention pooling mechanism to predict the sentence-level scores and optimize the component models with a multitask learning (MTL) framework. Extensive experiments carried out on a publicly-available benchmark dataset, viz. speechocean762, demonstrate the efficacy of our approach in relation to some cutting-edge baselines.

翻译：自动发音评估（APA）在计算机辅助发音训练（CAPT）中评估第二语言学习者口语熟练度时扮演着关键角色。然而，现有大多数方法的明显缺陷在于，它们将不同语音粒度的建模过程并行化，而未充分考虑这些粒度之间的层级与局部上下文关系。鉴于此，本文提出了一种新颖的层级化方法，用于实现多维度与多粒度的自动发音评估。具体而言：首先，引入超音位概念以探索第二语言使用者更细微的语义特征；其次，利用深度可分离卷积层更好地捕捉子词层面的局部上下文线索；最后，采用分数约束注意力池化机制预测句子级别得分，并通过多任务学习框架优化各组件模型。在公开基准数据集speechocean762上进行的大量实验证明，相较于若干前沿基线方法，本方法具有显著有效性。