Existing cross-lingual transfer (CLT) prompting methods are only concerned with monolingual demonstration examples in the source language. In this paper, we propose In-CLT, a novel cross-lingual transfer prompting method that leverages both source and target languages to construct the demonstration examples. We conduct comprehensive evaluations on multilingual benchmarks, focusing on question answering tasks. Experiment results show that In-CLT prompt not only improves multilingual models' cross-lingual transferability, but also demonstrates remarkable unseen language generalization ability. In-CLT prompting, in particular, improves model performance by 10 to 20\% points on average when compared to prior cross-lingual transfer approaches. We also observe the surprising performance gain on the other multilingual benchmarks, especially in reasoning tasks. Furthermore, we investigate the relationship between lexical similarity and pre-training corpora in terms of the cross-lingual transfer gap.

翻译：现有跨语言迁移提示方法仅关注源语言中的单语展示样例。本文提出In-CLT——一种新颖的跨语言迁移提示方法，该方法同时利用源语言和目标语言构建展示样例。我们在多语言基准上开展了全面评估，重点关注问答任务。实验结果表明，In-CLT提示不仅提升了多语言模型的跨语言迁移能力，还展现出显著的非训练语种泛化性能。与先前跨语言迁移方法相比，In-CLT提示平均可提升模型性能10至20个百分点。我们还在其他多语言基准上观察到性能提升，尤其在推理任务中。此外，我们探讨了词汇相似度与预训练语料库在跨语言迁移差距方面的关联。