Any piece of knowledge is usually expressed in one or a handful of natural languages on the web or in any large corpus. Large Language Models (LLMs) act as a bridge by acquiring knowledge from a source language and making it accessible when queried from target languages. Prior research has pointed to a cross-lingual gap, viz., a drop in accuracy when the knowledge is queried in a target language compared to when the query is in the source language. Existing research has rationalized divergence in latent representations in source and target languages as the source of cross-lingual gap. In this work, we take an alternative view and hypothesize that the variance of responses in the target language is the main cause of this gap. For the first time, we formalize the cross-lingual gap in terms of bias-variance decomposition. We present extensive experimental evidence which support proposed formulation and hypothesis. We then reinforce our hypothesis through multiple inference-time interventions that control the variance and reduce the cross-lingual gap. We demonstrate a simple prompt instruction to reduce the response variance, which improved target accuracy by 20-25% across different models.

翻译：任何知识片段通常在网络或大型语料库中以一种或少数几种自然语言表达。大型语言模型（LLMs）通过从源语言获取知识，并在目标语言查询时提供访问，起到了桥梁作用。先前研究指出了跨语言鸿沟的存在，即与源语言查询相比，用目标语言查询知识时准确率下降。现有研究将源语言与目标语言潜在表征的差异归因为跨语言鸿沟的根源。在本研究中，我们提出另一种观点，假设目标语言响应的方差是此鸿沟的主要成因。我们首次通过偏差-方差分解的形式化方法定义跨语言鸿沟。我们提供了大量实验证据支持所提出的形式化框架与假设。随后，我们通过多种控制方差、减少跨语言鸿沟的推理时干预措施进一步验证假设。我们展示了一种简单的提示指令来降低响应方差，该指令在不同模型中将目标语言准确率提升了20-25%。