Even with several advancements in multilingual modeling, it is challenging to recognize multiple languages using a single neural model, without knowing the input language and most multilingual models assume the availability of the input language. In this work, we propose a novel bilingual end-to-end (E2E) modeling approach, where a single neural model can recognize both languages and also support switching between the languages, without any language input from the user. The proposed model has shared encoder and prediction networks, with language-specific joint networks that are combined via a self-attention mechanism. As the language-specific posteriors are combined, it produces a single posterior probability over all the output symbols, enabling a single beam search decoding and also allowing dynamic switching between the languages. The proposed approach outperforms the conventional bilingual baseline with 13.3%, 8.23% and 1.3% word error rate relative reduction on Hindi, English and code-mixed test sets, respectively.

翻译：尽管多语言建模取得了诸多进展，但在未知输入语言的情况下，使用单一神经网络模型识别多种语言仍具挑战性，且大多数多语言模型都假设已知输入语言。本文提出一种新颖的双语端到端建模方法，该方法无需用户提供任何语言信息，单一神经网络模型即可识别两种语言并支持语言间切换。所提模型共享编码器和预测网络，通过自注意力机制将语言特定的联合网络进行组合。由于语言特定的后验概率被合并，系统在所有输出符号上生成单一后验概率，从而支持单波束搜索解码并实现语言间的动态切换。实验结果表明，与常规双语基线相比，所提方法在印地语、英语和混合语测试集上分别取得13.3%、8.23%和1.3%的词错误率相对降低。