Multilingual pretrained language models (MPLMs) have demonstrated substantial performance improvements in zero-shot cross-lingual transfer across various natural language understanding tasks by finetuning MPLMs on task-specific labelled data of a source language (e.g. English) and evaluating on a wide range of target languages. Recent studies show that prompt-based finetuning surpasses regular finetuning in few-shot scenarios. However, the exploration of prompt-based learning in multilingual tasks remains limited. In this study, we propose the ProFiT pipeline to investigate the cross-lingual capabilities of Prompt-based Finetuning. We conduct comprehensive experiments on diverse cross-lingual language understanding tasks (sentiment classification, paraphrase identification, and natural language inference) and empirically analyze the variation trends of prompt-based finetuning performance in cross-lingual transfer across different few-shot and full-data settings. Our results reveal the effectiveness and versatility of prompt-based finetuning in cross-lingual language understanding. Our findings indicate that prompt-based finetuning outperforms vanilla finetuning in full-data scenarios and exhibits greater advantages in few-shot scenarios, with different performance patterns dependent on task types. Additionally, we analyze underlying factors such as language similarity and pretraining data size that impact the cross-lingual performance of prompt-based finetuning. Overall, our work provides valuable insights into the cross-lingual prowess of prompt-based finetuning.

翻译：多语言预训练语言模型（MPLMs）通过在源语言（例如英语）的任务特定标注数据上微调，并在多种目标语言上评估，已在零样本跨语言迁移中展现出显著性能提升。近期研究表明，基于提示的微调在少样本场景下优于常规微调。然而，对多语言任务中基于提示学习的探索仍很有限。本研究提出ProFiT流程，以探究基于提示微调的跨语言能力。我们在多样化的跨语言语言理解任务（情感分类、释义识别和自然语言推理）上进行了全面实验，并实证分析了基于提示微调在不同少样本和全数据设置下跨语言迁移中的性能变化趋势。我们的结果揭示了基于提示微调在跨语言语言理解中的有效性和多适应性。研究发现表明，基于提示微调在全数据场景下优于普通微调，在少样本场景下展现出更大优势，且不同任务类型呈现不同的性能模式。此外，我们分析了影响基于提示微调跨语言性能的潜在因素，如语言相似性和预训练数据规模。总体而言，我们的工作为基于提示微调的跨语言能力提供了有价值的见解。