We expose the limitation of modular multilingual language models (MLMs) in multilingual inference scenarios with unknown languages. Existing evaluations of modular MLMs exclude the involvement of language identification (LID) modules, which obscures the performance of real-case multilingual scenarios of modular MLMs. In this work, we showcase the effect of adding LID on the multilingual evaluation of modular MLMs and provide discussions for closing the performance gap of caused by the pipelined approach of LID and modular MLMs.

翻译：我们揭示了模块化多语言语言模型（MLMs）在涉及未知语言的多语言推理场景中存在的局限。现有对模块化MLMs的评估排除了语言识别（LID）模块的参与，这掩盖了模块化MLMs在实际多语言场景中的性能表现。本研究展示了在模块化MLMs的多语言评估中加入LID模块所产生的效果，并针对LID与模块化MLMs采用流水线方式处理所导致的性能差距，提供了弥合该差距的讨论方案。