In-context learning (ICL) leverages in-context examples as prompts for the predictions of Large Language Models (LLMs). These prompts play a crucial role in achieving strong performance. However, the selection of suitable prompts from a large pool of labeled examples often entails significant annotation costs. To address this challenge, we propose \textbf{Sub-SA} (\textbf{Sub}modular \textbf{S}elective \textbf{A}nnotation), a submodule-based selective annotation method. The aim of Sub-SA is to reduce annotation costs while improving the quality of in-context examples and minimizing the time consumption of the selection process. In Sub-SA, we design a submodular function that facilitates effective subset selection for annotation and demonstrates the characteristics of monotonically and submodularity from the theoretical perspective. Specifically, we propose \textbf{RPR} (\textbf{R}eward and \textbf{P}enalty \textbf{R}egularization) to better balance the diversity and representativeness of the unlabeled dataset attributed to a reward term and a penalty term, respectively. Consequently, the selection for annotations can be effectively addressed with a simple yet effective greedy search algorithm based on the submodular function. Finally, we apply the similarity prompt retrieval to get the examples for ICL.

翻译：上下文学习（ICL）利用上下文示例作为大型语言模型（LLM）预测的提示。这些提示对于实现强劲性能至关重要。然而，从大量标注示例中筛选合适的提示通常需要高昂的标注成本。为应对这一挑战，我们提出\textbf{Sub-SA}（基于\textbf{子模}的\textbf{选择性}标\textbf{注}方法），这是一种基于子模函数的选择性标注方法。Sub-SA旨在降低标注成本的同时提升上下文示例质量，并最小化选择过程的时间消耗。在Sub-SA中，我们设计了一个子模函数以促进有效的标注子集选择，并从理论角度证明了其单调性与子模性。具体而言，我们提出\textbf{RPR}（\textbf{奖}励与\textbf{惩}罚\textbf{正则化}），通过奖励项与惩罚项分别优化未标注数据集的多样性与代表性。基于该子模函数，我们采用简单高效的最优搜索算法即可有效解决标注选择问题。最终，通过相似性提示检索获取用于ICL的示例。