Advances in machine learning have made it possible to perform various text and speech processing tasks, such as automatic speech recognition (ASR), in an end-to-end (E2E) manner. E2E approaches utilizing pre-trained models are gaining attention for conserving training data and resources. However, most of their applications in ASR involve only one of either a pre-trained speech or a language model. This paper proposes integrating a pre-trained speech representation model and a large language model (LLM) for E2E ASR. The proposed model enables the optimization of the entire ASR process, including acoustic feature extraction and acoustic and language modeling, by combining pre-trained models with a bridge network and also enables the application of remarkable developments in LLM utilization, such as parameter-efficient domain adaptation and inference optimization. Experimental results demonstrate that the proposed model achieves a performance comparable to that of modern E2E ASR models by utilizing powerful pre-training models with the proposed integrated approach.

翻译：机器学习的发展使得以端到端（E2E）方式执行各种文本和语音处理任务（例如自动语音识别（ASR））成为可能。利用预训练模型的端到端方法因能节省训练数据和资源而日益受到关注。然而，它们在ASR中的大多数应用仅涉及预训练语音模型或语言模型中的一种。本文提出将预训练语音表征模型与大型语言模型（LLM）相结合，用于端到端ASR。所提出的模型通过将预训练模型与一个桥接网络相结合，实现了对整个ASR过程（包括声学特征提取以及声学和语言建模）的优化，同时也使得LLM利用方面的显著进展（例如参数高效的领域自适应和推理优化）得以应用。实验结果表明，通过利用强大的预训练模型并结合所提出的集成方法，该模型实现了与现代端到端ASR模型相媲美的性能。