Large pretrained language models (LMs) have shown impressive In-Context Learning (ICL) ability, where the model learns to do an unseen task via a prompt consisting of input-output examples as the demonstration, without any parameter updates. The performance of ICL is highly dominated by the quality of the selected in-context examples. However, previous selection methods are mostly based on simple heuristics, leading to sub-optimal performance. In this work, we formulate in-context example selection as a subset selection problem. We propose CEIL (Compositional Exemplars for In-context Learning), which is instantiated by Determinantal Point Processes (DPPs) to model the interaction between the given input and in-context examples, and optimized through a carefully-designed contrastive learning objective to obtain preference from LMs. We validate CEIL on 12 classification and generation datasets from 7 distinct NLP tasks, including sentiment analysis, paraphrase detection, natural language inference, commonsense reasoning, open-domain question answering, code generation, and semantic parsing. Extensive experiments demonstrate not only the state-of-the-art performance but also the transferability and compositionality of CEIL, shedding new light on effective and efficient in-context learning. Our code is released at https://github.com/HKUNLP/icl-ceil.

翻译：大型预先培训的语言模型(LMS)已经表现出令人印象深刻的 Incontext Learning (ICL) 能力, 模型通过由输入- 输出示例组成的快速演示, 学习完成一项看不见的任务, 无需更新任何参数。 ILLL 的性能高度受选定的文文本示例质量的制约。 但是, 以前的选择方法大多基于简单的超自然学, 导致亚优性性能。 在这项工作中, 我们将同文文本示例选择作为一个子选择问题。 我们建议 CEIL (Incontical Explanderers for Incontext Learning) (In Celicial EIL (Compositional-Explace Explainations for Inconomications), 由 Deptminate- productions- Codeport and semmantical C-Lical contravely) 。 我们的新的CELILIL( Procial- Providuction) 并仅展示了我们新的状态解答、 Excial- Expartial Excial Excidutional- Excidudustrational eximation.</s>