In-Context Learning (ICL) is an important paradigm for adapting Large Language Models (LLMs) to downstream tasks through a few demonstrations. Despite the great success of ICL, the limitation of the demonstration number may lead to demonstration bias, i.e. the input-label mapping induced by LLMs misunderstands the task's essence. Inspired by human experience, we attempt to mitigate such bias through the perspective of the inter-demonstration relationship. Specifically, we construct Comparable Demonstrations (CDs) by minimally editing the texts to flip the corresponding labels, in order to highlight the task's essence and eliminate potential spurious correlations through the inter-demonstration comparison. Through a series of experiments on CDs, we find that (1) demonstration bias does exist in LLMs, and CDs can significantly reduce such bias; (2) CDs exhibit good performance in ICL, especially in out-of-distribution scenarios. In summary, this study explores the ICL mechanisms from a novel perspective, providing a deeper insight into the demonstration selection strategy for ICL.

翻译：上下文学习（ICL）是通过少量示例使大型语言模型（LLMs）适应下游任务的重要范式。尽管ICL取得了巨大成功，但示例数量的限制可能导致示例偏差，即LLMs诱导的输入-标签映射误解了任务本质。受人类经验启发，我们尝试通过示例间关系的视角来缓解这种偏差。具体而言，我们通过最小化编辑文本来翻转相应标签，从而构建可比较示例（CDs），旨在通过示例间比较突出任务本质并消除潜在的虚假相关性。通过一系列关于CDs的实验，我们发现：（1）LLMs中确实存在示例偏差，而CDs能显著降低此类偏差；（2）CDs在ICL中表现良好，尤其在分布外场景中。综上所述，本研究从新视角探索ICL机制，为ICL的示例选择策略提供了更深入的见解。