The INTERSPEECH 2025 Challenge on Multilingual Conversational Speech Language Models (MLC-SLM) promotes multilingual conversational ASR with large language models (LLMs). Our previous SHNU-mASR system adopted a competitive parallel-speech-encoder architecture that integrated Whisper and mHuBERT with an LLM. However, it faced two challenges: simple feature concatenation may not fully exploit complementary information, and the performance gap between LLM-based ASR and end-to-end(E2E) encoder-decoder ASR remained unexplored. In this work, we present an enhanced LLM-based ASR framework that combines fine-tuned Whisper and mHuBERT encoders with an LLM to enrich speech representations. We first evaluate E2E Whisper models with LoRA and full fine-tuning on the MLC-SLM ASR task, and then propose cross-attention-based fusion mechanisms for the parallel-speech-encoder. On the official evaluation set of the MLC-SLM Challenge, our system achieves a CER/WER of 10.69%, ranking on par with the top-ranked Track 1 systems, even though it uses only 1,500 hours of baseline training data compared with their large-scale training sets. Nonetheless, we find that our final LLM-based ASR still does not match the performance of a fine-tuned E2E Whisper model, providing valuable empirical guidance for future Speech-LLM design. Our code is publicly available at https://github.com/1535176727/MLC-SLM.

翻译：INTERSPEECH 2025多语言对话语音语言模型挑战赛旨在推动基于大语言模型的多语言对话语音识别研究。我们先前提出的SHNU-mASR系统采用了一种具有竞争力的并行语音编码器架构，将Whisper与mHuBERT与大语言模型相结合。然而，该系统面临两大挑战：简单的特征拼接可能无法充分利用互补信息，且基于LLM的语音识别与端到端编码器-解码器语音识别之间的性能差距尚未得到充分探究。本研究提出一种增强的基于LLM的语音识别框架，通过整合微调后的Whisper与mHuBERT编码器及LLM来丰富语音表征。我们首先评估了采用LoRA与全参数微调的端到端Whisper模型在MLC-SLM语音识别任务上的表现，随后为并行语音编码器提出了基于交叉注意力的融合机制。在MLC-SLM挑战赛官方评估集上，我们的系统取得了10.69%的字错误率/词错误率，与排名前列的赛道一系统性能相当，尽管我们仅使用了1,500小时的基线训练数据，而其他系统采用了大规模训练集。然而，我们发现最终基于LLM的语音识别系统仍未能超越经过微调的端到端Whisper模型，这为未来Speech-LLM的设计提供了重要的实证参考。代码已公开于https://github.com/1535176727/MLC-SLM。