Multilingual pretrained language models serve as repositories of multilingual factual knowledge. Nevertheless, a substantial performance gap of factual knowledge probing exists between high-resource languages and low-resource languages, suggesting limited implicit factual knowledge transfer across languages in multilingual pretrained language models. This paper investigates the feasibility of explicitly transferring relatively rich factual knowledge from English to non-English languages. To accomplish this, we propose two parameter-free $\textbf{L}$anguage $\textbf{R}$epresentation $\textbf{P}$rojection modules (LRP2). The first module converts non-English representations into English-like equivalents, while the second module reverts English-like representations back into representations of the corresponding non-English language. Experimental results on the mLAMA dataset demonstrate that LRP2 significantly improves factual knowledge retrieval accuracy and facilitates knowledge transferability across diverse non-English languages. We further investigate the working mechanism of LRP2 from the perspectives of representation space and cross-lingual knowledge neuron.

翻译：多语言预训练语言模型是多语言事实知识的存储库。然而，在高资源语言与低资源语言之间，事实知识探测的性能存在显著差距，这表明多语言预训练语言模型在跨语言方面存在有限的隐式事实知识迁移能力。本文探讨了将相对丰富的事实知识从英语显式迁移到非英语语言的可行性。为此，我们提出了两个无参数的$\textbf{语言表征投影模块}$（LRP2）。第一个模块将非英语表征转换为类似英语的等价表征，而第二个模块则将类似英语的表征还原为对应非英语语言的表征。在mLAMA数据集上的实验结果表明，LRP2显著提高了事实知识检索的准确性，并促进了跨多种非英语语言的知识可迁移性。我们进一步从表征空间和跨语言知识神经元的角度探究了LRP2的工作机制。