Few-shot classification has made great strides due to foundation models that, through priming and prompting, are highly effective few-shot learners. However, this approach has high variance both across different sets of few shots (data selection) and across different finetuning runs (run variability). This is problematic not only because it impedes the fair comparison of different approaches, but especially because it makes few-shot learning too unreliable for many real-world applications. To alleviate these issues, we make two contributions for more stable and effective few-shot learning: First, we propose novel ensembling methods and show that they substantially reduce run variability. Second, we introduce a new active learning (AL) criterion for data selection and present the first AL-based approach specifically tailored towards prompt-based learning. In our experiments, we show that our combined method, MEAL (Multiprompt finetuning and prediction Ensembling with Active Learning), improves overall performance of prompt-based finetuning by 2.3 points on five diverse tasks. We publicly share our code and data splits in https://github.com/akoksal/MEAL.

翻译：少样本分类因基础模型的发展取得了显著进展，这些模型通过预置提示和即时提示成为高效的少样本学习器。然而，该方法在不同少样本集（数据选择）和不同微调运行（运行变异性）中均存在高方差问题。这不仅阻碍了不同方法的公平比较，更关键的是导致少样本学习因可靠性不足而难以应用于实际场景。为解决这些问题，我们提出两项贡献以实现更稳定有效的少样本学习：首先，提出新型集成方法，实验表明该方法能显著降低运行变异性；其次，针对数据选择引入新的主动学习（AL）准则，并首次提出专门面向提示学习的AL方法。实验证明，我们的组合方法MEAL（基于主动学习的多提示微调与预测集成）在五个不同任务上使提示微调的整体性能提升2.3个百分点。我们在https://github.com/akoksal/MEAL 公开共享代码与数据分割。