In few-shot learning, such as meta-learning, few-shot fine-tuning or in-context learning, the limited number of samples used to train a model have a significant impact on the overall success. Although a large number of sample selection strategies exist, their impact on the performance of few-shot learning is not extensively known, as most of them have been so far evaluated in typical supervised settings only. In this paper, we thoroughly investigate the impact of 20 sample selection strategies on the performance of 5 few-shot learning approaches over 8 image and 6 text datasets. In addition, we propose a new method for automatic combination of sample selection strategies (ACSESS) that leverages the strengths and complementary information of the individual strategies. The experimental results show that our method consistently outperforms the individual selection strategies, as well as the recently proposed method for selecting support examples for in-context learning. We also show a strong modality, dataset and approach dependence for the majority of strategies as well as their dependence on the number of shots - demonstrating that the sample selection strategies play a significant role for lower number of shots, but regresses to random selection at higher number of shots.

翻译：在小样本学习（如元学习、小样本微调或上下文学习）中，用于训练模型的样本数量有限，这对整体成功率具有显著影响。尽管存在大量样本选择策略，但它们对小样本学习性能的影响尚未被广泛了解，因为到目前为止，大多数策略仅在典型的监督学习设置下进行了评估。本文深入研究了20种样本选择策略对5种小样本学习方法在8个图像数据集和6个文本数据集上的性能影响。此外，我们提出了一种新的自动组合样本选择策略方法（ACSESS），该方法利用了单个策略的优势和互补信息。实验结果表明，我们的方法在性能上始终优于单个选择策略以及最近提出的用于上下文学习支持样本选择的方法。我们还发现，大多数策略对模态、数据集和方法具有强烈的依赖性，并且这些策略依赖于样本数量——这表明样本选择策略在样本数量较少时发挥重要作用，但在样本数量较多时退化为随机选择。