Zero-Shot Cross-lingual Transfer (ZS-XLT) utilizes a model trained in a source language to make predictions in another language, often with a performance loss. To alleviate this, additional improvements can be achieved through subsequent adaptation using examples in the target language. In this paper, we exploit In-Context Tuning (ICT) for One-Shot Cross-lingual transfer in the classification task by introducing In-Context Cross-lingual Transfer (IC-XLT). The novel concept involves training a model to learn from context examples and subsequently adapting it during inference to a target language by prepending a One-Shot context demonstration in that language. Our results show that IC-XLT successfully leverages target-language examples to improve the cross-lingual capabilities of the evaluated mT5 model, outperforming prompt-based models in the Zero and Few-shot scenarios adapted through fine-tuning. Moreover, we show that when source-language data is limited, the fine-tuning framework employed for IC-XLT performs comparably to prompt-based fine-tuning with significantly more training data in the source language.

翻译：零样本跨语言迁移（ZS-XLT）利用在源语言上训练的模型对另一种语言进行预测，但通常会出现性能损失。为缓解这一问题，可通过在目标语言中使用示例进行后续适应来获得额外改进。本文通过引入上下文跨语言迁移（IC-XLT），在分类任务中探索基于上下文调优（ICT）的单样本跨语言迁移方法。该核心思想是训练模型从上下文示例中学习，并在推理阶段通过附加目标语言的单样本上下文示范，使其适应目标语言。实验表明，IC-XLT能够有效利用目标语言示例提升所评估的mT5模型的跨语言能力，在零样本和少样本场景中均优于通过微调适配的提示模型。此外，我们发现在源语言数据有限的情况下，IC-XLT所采用的微调框架的性能可与使用大量源语言训练数据的提示微调方法相媲美。