In-context learning (ICL) suffers from oversensitivity to the prompt, making it unreliable in real-world scenarios. We study the sensitivity of ICL with respect to multiple perturbation types. First, we find that label bias obscures the true sensitivity, and therefore prior work may have significantly underestimated ICL sensitivity. Second, we observe a strong negative correlation between ICL sensitivity and accuracy: predictions sensitive to perturbations are less likely to be correct. Motivated by these findings, we propose \textsc{SenSel}, a few-shot selective prediction method that abstains from sensitive predictions. Experiments on ten classification datasets show that \textsc{SenSel} consistently outperforms two commonly used confidence-based and entropy-based baselines on abstention decisions.

翻译：上下文学习（ICL）对提示的过度敏感性问题，导致其在现实场景中缺乏可靠性。本文研究了ICL对多种扰动类型的敏感性。首先，我们发现标签偏差掩盖了真实的敏感性，因此先前的研究可能严重低估了ICL的敏感性。其次，我们观察到ICL敏感性与准确性之间存在强负相关：对扰动敏感的预测更不可能正确。基于这些发现，我们提出 \textsc{SenSel}——一种选择性预测方法，通过放弃敏感预测来提升可靠性。在十个分类数据集上的实验表明，\textsc{SenSel} 在放弃决策任务上持续优于两种常用的基于置信度和基于熵的基线方法。