Recent advancements of in-context learning (ICL) show language models can significantly improve their performance when demonstrations are provided. However, little attention has been paid to model calibration and prediction confidence of ICL in cross-lingual scenarios. To bridge this gap, we conduct a thorough analysis of ICL for cross-lingual sentiment classification. Our findings suggest that ICL performs poorly in cross-lingual scenarios, exhibiting low accuracy and presenting high calibration errors. In response, we propose a novel approach, N2C2, which employs a -nearest neighbors augmented classifier for prediction confidence calibration. N2C2 narrows the prediction gap by leveraging a datastore of cached few-shot instances. Specifically, N2C2 integrates the predictions from the datastore and incorporates confidence-aware distribution, semantically consistent retrieval representation, and adaptive neighbor combination modules to effectively utilize the limited number of supporting instances. Evaluation on two multilingual sentiment classification datasets demonstrates that N2C2 outperforms traditional ICL. It surpasses fine tuning, prompt tuning and recent state-of-the-art methods in terms of accuracy and calibration errors.

翻译：上下文学习（ICL）的最新进展表明，语言模型在提供演示样例时能显著提升性能。然而，在跨语言场景下，ICL的模型校准与预测置信度问题尚未得到充分关注。为填补这一空白，我们对跨语言情感分类任务中的ICL进行了系统分析。研究发现，ICL在跨语言场景中表现欠佳，存在准确率低、校准误差高的问题。为此，我们提出了一种新方法N2C2，该方法采用基于k最近邻的增强分类器进行预测置信度校准。N2C2通过利用缓存的少样本实例数据存储库来缩小预测差距。具体而言，N2C2整合了来自数据存储库的预测结果，并融合了置信度感知分布模块、语义一致性检索表示模块和自适应邻居组合模块，以有效利用有限数量的支持实例。在两个多语言情感分类数据集上的评估表明，N2C2优于传统ICL方法，在准确率和校准误差方面均超越了微调、提示调优及近期最先进的方法。