In-Context Learning (ICL) is an important paradigm for adapting Large Language Models (LLMs) to downstream tasks through a few demonstrations. Despite the great success of ICL, the limitation of the demonstration number may lead to demonstration bias, i.e. the input-label mapping induced by LLMs misunderstands the task's essence. Inspired by human experience, we attempt to mitigate such bias through the perspective of the inter-demonstration relationship. Specifically, we construct Comparable Demonstrations (CDs) by minimally editing the texts to flip the corresponding labels, in order to highlight the task's essence and eliminate potential spurious correlations through the inter-demonstration comparison. Through a series of experiments on CDs, we find that (1) demonstration bias does exist in LLMs, and CDs can significantly reduce such bias; (2) CDs exhibit good performance in ICL, especially in out-of-distribution scenarios. In summary, this study explores the ICL mechanisms from a novel perspective, providing a deeper insight into the demonstration selection strategy for ICL.

翻译：上下文学习（ICL）是通过少量示范使大型语言模型（LLMs）适应下游任务的重要范式。尽管ICL取得了巨大成功，但示范数量的限制可能导致示范偏差，即LLM诱导的输入-标签映射误解了任务本质。受人类经验启发，我们试图通过示范间关系的视角来缓解此类偏差。具体而言，我们通过最小化文本编辑来翻转对应标签，构建可比较示范（CDs），从而通过示范间对比突出任务本质并消除潜在虚假关联。通过一系列基于CDs的实验，我们发现：（1）示范偏差确实存在于LLMs中，而CDs能显著降低此类偏差；（2）CDs在ICL中展现出良好性能，尤其是在分布外场景中。总之，本研究从新视角探索ICL机制，为ICL的示范选择策略提供了更深入的见解。