In-context learning is the paradigm that adapts large language models to downstream tasks by providing a few examples. Few-shot selection -- selecting appropriate examples for each test instance separately -- is important for in-context learning. In this paper, we propose Skill-KNN, a skill-based few-shot selection method for in-context learning. The key advantages of Skill-KNN include: (1) it addresses the problem that existing methods based on pre-trained embeddings can be easily biased by surface natural language features that are not important for the target task; (2) it does not require training or fine-tuning of any models, making it suitable for frequently expanding or changing example banks. The key insight is to optimize the inputs fed into the embedding model, rather than tuning the model itself. Technically, Skill-KNN generates the skill-based descriptions for each test case and candidate example by utilizing a pre-processing few-shot prompting, thus eliminating unimportant surface features. Experimental results across five cross-domain semantic parsing datasets and six backbone models show that Skill-KNN significantly outperforms existing methods.

翻译：上下文学习是一种通过提供少量示例使大型语言模型适配下游任务的范式。小样本选择——为每个测试实例分别选择合适的示例——对上下文学习至关重要。本文提出Skill-KNN，一种基于技能的上下文学习小样本选择方法。Skill-KNN的主要优势包括：(1) 解决了现有基于预训练嵌入的方法易受与目标任务无关的表面自然语言特征干扰的问题；(2) 无需训练或微调任何模型，适用于示例库频繁扩展或更新的场景。其核心思想是优化输入嵌入模型的输入内容，而非调整模型本身。在技术实现上，Skill-KNN通过预处理小样本提示为每个测试用例和候选示例生成基于技能的描述，从而消除无关的表面特征。在五个跨领域语义解析数据集和六个骨干模型上的实验结果表明，Skill-KNN显著优于现有方法。