Personalization in multi-turn dialogs has been a long standing challenge for end-to-end automatic speech recognition (E2E ASR) models. Recent work on contextual adapters has tackled rare word recognition using user catalogs. This adaptation, however, does not incorporate an important cue, the dialog act, which is available in a multi-turn dialog scenario. In this work, we propose a dialog act guided contextual adapter network. Specifically, it leverages dialog acts to select the most relevant user catalogs and creates queries based on both -- the audio as well as the semantic relationship between the carrier phrase and user catalogs to better guide the contextual biasing. On industrial voice assistant datasets, our model outperforms both the baselines - dialog act encoder-only model, and the contextual adaptation, leading to the most improvement over the no-context model: 58% average relative word error rate reduction (WERR) in the multi-turn dialog scenario, in comparison to the prior-art contextual adapter, which has achieved 39% WERR over the no-context model.

翻译：多轮对话中的个性化一直是端到端自动语音识别（E2E ASR）模型面临的长期挑战。近期关于上下文适配器的工作利用用户目录处理了罕见词识别问题。然而，这种适配并未纳入一个重要的线索——对话行为，而该线索在多轮对话场景中是可获取的。在本工作中，我们提出了一种对话行为引导的上下文适配器网络。具体而言，它利用对话行为选择最相关的用户目录，并基于音频以及承载短语与用户目录之间的语义关系创建查询，以更好地引导上下文偏置。在工业级语音助手数据集上，我们的模型优于两个基线——仅对话行为编码器模型和上下文适配模型，相比无上下文模型实现了最大改进：在多轮对话场景中，平均相对词错误率降低（WERR）达58%，而先前技术的上下文适配器相比无上下文模型仅实现了39%的WERR。