Prompts are crucial to large language models as they provide context information such as topic or logical relationships. Inspired by this, we propose PromptASR, a framework that integrates prompts in end-to-end automatic speech recognition (E2E ASR) systems to achieve contextualized ASR with controllable style of transcriptions. Specifically, a dedicated text encoder encodes the text prompts and the encodings are injected into the speech encoder by cross-attending the features from two modalities. When using the ground truth text from preceding utterances as content prompt, the proposed system achieves 21.9% and 6.8% relative word error rate reductions on a book reading dataset and an in-house dataset compared to a baseline ASR system. The system can also take word-level biasing lists as prompt to improve recognition accuracy on rare words. An additional style prompt can be given to the text encoder and guide the ASR system to output different styles of transcriptions. The code is available at icefall.

翻译：提示（Prompt）对大型语言模型至关重要，因其能提供主题或逻辑关系等上下文信息。受此启发，我们提出PromptASR框架——一种将提示机制融入端到端自动语音识别（E2E ASR）系统的方法，旨在实现具有转录文本风格可控性的上下文感知语音识别。具体而言，专用文本编码器对文本提示进行编码，并通过跨模态特征交叉注意力机制将编码注入语音编码器。当使用前序语句的真实文本作为内容提示时，所提系统在图书朗读数据集和内部数据集上相较于基线ASR系统分别实现了21.9%和6.8%的相对词错误率降幅。该系统还能将词级偏置列表作为提示输入，以提升稀有词的识别准确率。额外的风格提示可输入文本编码器，引导ASR系统输出不同风格的转录文本。相关代码已在icefall平台开源。