In-context learning (ICL) unfolds as large language models become capable of inferring test labels conditioned on a few labeled samples without any gradient update. ICL-enabled large language models provide a promising step forward toward bypassing recurrent annotation costs in a low-resource setting. Yet, only a handful of past studies have explored ICL in a cross-lingual setting, in which the need for transferring label-knowledge from a high-resource language to a low-resource one is immensely crucial. To bridge the gap, we provide the first in-depth analysis of ICL for cross-lingual text classification. We find that the prevalent mode of selecting random input-label pairs to construct the prompt-context is severely limited in the case of cross-lingual ICL, primarily due to the lack of alignment in the input as well as the output spaces. To mitigate this, we propose a novel prompt construction strategy -- Cross-lingual In-context Source-Target Alignment (X-InSTA). With an injected coherence in the semantics of the input examples and a task-based alignment across the source and target languages, X-InSTA is able to outperform random prompt selection by a large margin across three different tasks using 44 different cross-lingual pairs.

翻译：上下文学习（ICL）指大语言模型无需梯度更新，仅凭少量标注样本即可推断测试标签的能力。支持ICL的大语言模型为在低资源场景中绕过重复标注成本提供了有前景的解决方案。然而，仅有少数过往研究探索了跨语言环境下的ICL——而在该场景中，将标签知识从高资源语言迁移至低资源语言的需求极为关键。为弥补这一缺口，我们首次对跨语言文本分类中的ICL进行了深入分析。研究发现，当前构建提示上下文时主流的随机选择输入-标签对模式，在跨语言ICL中存在严重局限，主要源于输入与输出空间均缺乏对齐性。为此，我们提出了一种新颖的提示构建策略——跨语言上下文源-目标对齐（X-InSTA）。通过注入输入示例语义的一致性以及源语言与目标语言间的任务级对齐，X-InSTA在44个不同跨语言对的三个任务中，其性能均大幅优于随机提示选择方法。