Large language models (LLMs) have exhibited considerable cross-lingual generalization abilities, whereby they implicitly transfer knowledge across languages. However, the transfer is not equally successful for all languages, especially for low-resource ones, which poses an ongoing challenge. It is unclear whether we have reached the limits of implicit cross-lingual generalization and if explicit knowledge transfer is viable. In this paper, we investigate the potential for explicitly aligning conceptual correspondence between languages to enhance cross-lingual generalization. Using the syntactic aspect of language as a testbed, our analyses of 43 languages reveal a high degree of alignability among the spaces of structural concepts within each language for both encoder-only and decoder-only LLMs. We then propose a meta-learning-based method to learn to align conceptual spaces of different languages, which facilitates zero-shot and few-shot generalization in concept classification and also offers insights into the cross-lingual in-context learning phenomenon. Experiments on syntactic analysis tasks show that our approach achieves competitive results with state-of-the-art methods and narrows the performance gap between languages, particularly benefiting those with limited resources.

翻译：大型语言模型（LLMs）展现出显著的跨语言泛化能力，能够隐式地在语言间迁移知识。然而，这种迁移对所有语言（尤其是低资源语言）并非同样成功，这构成了一项持续挑战。目前尚不清楚我们是否已达到隐式跨语言泛化的极限，以及显式知识迁移是否可行。本文研究了显式对齐语言间概念对应关系以增强跨语言泛化的潜力。以语言的句法层面为测试平台，我们对43种语言的分析揭示，在仅编码器和仅解码器LLMs中，每种语言内部的结构概念空间均具有高度可对齐性。随后，我们提出一种基于元学习的方法来学习对齐不同语言的概念空间，该方法促进了零样本和少样本概念分类中的泛化，并为跨语言上下文学习现象提供了洞见。句法分析任务的实验表明，我们的方法取得了与最先进方法相当的竞争性结果，缩小了语言间的性能差距，尤其惠及资源有限的语言。