Demonstration selection is a practical bottleneck in in-context learning (ICL): under a tight prompt budget, accuracy can change substantially depending on which few-shot examples are included, yet selection must remain cheap enough to run per query over large candidate pools. We propose Meta-Sel, a lightweight supervised meta-learning approach for intent classification that learns a fast, interpretable scoring function for (candidate, query) pairs from labeled training data. Meta-Sel constructs a meta-dataset by sampling pairs from the training split and using class agreement as supervision, then trains a calibrated logistic regressor on two inexpensive meta-features: TF--IDF cosine similarity and a length-compatibility ratio. At inference time, the selector performs a single vectorized scoring pass over the full candidate pool and returns the top-k demonstrations, requiring no model fine-tuning, no online exploration, and no additional LLM calls. This yields deterministic rankings and makes the selection mechanism straightforward to audit via interpretable feature weights. Beyond proposing Meta-Sel, we provide a broad empirical study of demonstration selection, benchmarking 12 methods -- spanning prompt engineering baselines, heuristic selection, reinforcement learning, and influence-based approaches -- across four intent datasets and five open-source LLMs. Across this benchmark, Meta-Sel consistently ranks among the top-performing methods, is particularly effective for smaller models where selection quality can partially compensate for limited model capacity, and maintains competitive selection-time overhead.

翻译：演示选择是上下文学习（ICL）中的一个实际瓶颈：在有限的提示预算下，准确率会因包含的少样本示例而发生显著变化，但选择过程必须足够廉价，以便能在大型候选池上针对每个查询执行。我们提出元选择（Meta-Sel），一种用于意图分类的轻量级监督元学习方法，它从带标签的训练数据中学习一个快速、可解释的（候选，查询）对评分函数。Meta-Sel通过从训练分割中采样样本对并以类别一致性作为监督信号来构建元数据集，然后基于两个廉价的元特征——TF--IDF余弦相似度和长度兼容性比率——训练一个校准的逻辑回归器。在推理时，选择器对整个候选池执行一次向量化评分遍历并返回前k个演示，无需模型微调、在线探索或额外的LLM调用。这产生了确定性排序，并通过可解释的特征权重使选择机制易于审计。除了提出Meta-Sel，我们还对演示选择进行了广泛的实证研究，在四个意图数据集和五个开源LLM上对12种方法进行了基准测试——涵盖提示工程基线、启发式选择、强化学习和基于影响的方法。在此基准测试中，Meta-Sel始终位列性能最佳的方法之一，对于较小模型尤其有效（在这些模型中，选择质量可以部分弥补模型容量的不足），并保持了具有竞争力的选择时间开销。