Conventional ASR systems use frame-level phoneme posterior to conduct force-alignment~(FA) and provide timestamps, while end-to-end ASR systems especially AED based ones are short of such ability. This paper proposes to perform timestamp prediction~(TP) while recognizing by utilizing continuous integrate-and-fire~(CIF) mechanism in non-autoregressive ASR model - Paraformer. Foucing on the fire place bias issue of CIF, we conduct post-processing strategies including fire-delay and silence insertion. Besides, we propose to use scaled-CIF to smooth the weights of CIF output, which is proved beneficial for both ASR and TP task. Accumulated averaging shift~(AAS) and diarization error rate~(DER) are adopted to measure the quality of timestamps and we compare these metrics of proposed system and conventional hybrid force-alignment system. The experiment results over manually-marked timestamps testset show that the proposed optimization methods significantly improve the accuracy of CIF timestamps, reducing 66.7\% and 82.1\% of AAS and DER respectively. Comparing to Kaldi force-alignment trained with the same data, optimized CIF timestamps achieved 12.3\% relative AAS reduction.
翻译:传统ASR系统利用帧级音素后验进行强制对齐(FA)并提供时间戳,而端到端ASR系统(特别是基于AED的系统)缺乏这一能力。本文提出在非自回归ASR模型Paraformer中利用连续积分与触发(CIF)机制,在识别的同时进行时间戳预测(TP)。针对CIF的触发位置偏差问题,我们实施了包括触发延迟和静默插入在内的后处理策略。此外,我们提出使用缩放CIF(scaled-CIF)来平滑CIF输出的权重,这被证明对ASR和TP任务均有益。采用累积平均偏移(AAS)和话者分离错误率(DER)来衡量时间戳质量,并将所提系统与传统的混合强制对齐系统在这些指标上进行比较。在人工标注的时间戳测试集上的实验结果表明,所提优化方法显著提高了CIF时间戳的准确性,分别将AAS和DER降低了66.7%和82.1%。与使用相同数据训练的Kaldi强制对齐相比,优化后的CIF时间戳实现了12.3%的相对AAS降低。