This paper studies contextual biasing with Large Language Models (LLMs), where during second-pass rescoring additional contextual information is provided to a LLM to boost Automatic Speech Recognition (ASR) performance. We propose to leverage prompts for a LLM without fine tuning during rescoring which incorporate a biasing list and few-shot examples to serve as additional information when calculating the score for the hypothesis. In addition to few-shot prompt learning, we propose multi-task training of the LLM to predict both the entity class and the next token. To improve the efficiency for contextual biasing and to avoid exceeding LLMs' maximum sequence lengths, we propose dynamic prompting, where we select the most likely class using the class tag prediction, and only use entities in this class as contexts for next token prediction. Word Error Rate (WER) evaluation is performed on i) an internal calling, messaging, and dictation dataset, and ii) the SLUE-Voxpopuli dataset. Results indicate that biasing lists and few-shot examples can achieve 17.8% and 9.6% relative improvement compared to first pass ASR, and that multi-task training and dynamic prompting can achieve 20.0% and 11.3% relative WER improvement, respectively.

翻译：本文研究基于大型语言模型（LLM）的上下文偏置技术，在二次解码重打分阶段为LLM提供额外上下文信息以提升自动语音识别（ASR）性能。我们提出在重打分过程中无需微调即可利用LLM的提示（prompt）技术，通过整合偏置列表（biasing list）和少样本示例（few-shot examples），在计算假设得分时提供额外信息。除少样本提示学习外，我们还提出对LLM进行多任务训练以同时预测实体类别和下一个词元（token）。为提高上下文偏置效率并避免超出LLM最大序列长度限制，我们提出动态提示（dynamic prompting）方法：通过实体类别标签预测选择最可能的类别，仅将该类别实体作为下个词元预测的上下文。在以下数据集上进行了词错误率（WER）评估：(i) 内部通话、消息与听写数据集，(ii) SLUE-Voxpopuli数据集。结果表明，与单次解码ASR相比，偏置列表和少样本示例分别实现17.8%和9.6%的相对改善，而多任务训练和动态提示分别实现20.0%和11.3%的相对WER改善。