State-of-the-art few-shot learning (FSL) methods leverage prompt-based fine-tuning to obtain remarkable results for natural language understanding (NLU) tasks. While much of the prior FSL methods focus on improving downstream task performance, there is a limited understanding of the adversarial robustness of such methods. In this work, we conduct an extensive study of several state-of-the-art FSL methods to assess their robustness to adversarial perturbations. To better understand the impact of various factors towards robustness (or the lack of it), we evaluate prompt-based FSL methods against fully fine-tuned models for aspects such as the use of unlabeled data, multiple prompts, number of few-shot examples, model size and type. Our results on six GLUE tasks indicate that compared to fully fine-tuned models, vanilla FSL methods lead to a notable relative drop in task performance (i.e., are less robust) in the face of adversarial perturbations. However, using (i) unlabeled data for prompt-based FSL and (ii) multiple prompts flip the trend. We further demonstrate that increasing the number of few-shot examples and model size lead to increased adversarial robustness of vanilla FSL methods. Broadly, our work sheds light on the adversarial robustness evaluation of prompt-based FSL methods for NLU tasks.

翻译：最先进的少样本学习方法利用基于提示的微调，在自然语言理解任务中取得了显著成果。然而，尽管许多先前的少样本学习方法侧重于提升下游任务性能，但对此类方法的对抗鲁棒性理解有限。在本工作中，我们对几种最先进的少样本学习方法进行了广泛研究，评估其对对抗扰动的鲁棒性。为更好地理解不同因素对鲁棒性（或缺乏鲁棒性）的影响，我们针对未标注数据使用、多个提示、少样本示例数量、模型规模及类型等维度，将基于提示的少样本方法与全微调模型进行了对比评估。在六个GLUE任务上的结果表明：与全微调模型相比，原始少样本方法在面对对抗扰动时任务性能呈现显著相对下降（即鲁棒性较差）。然而，使用（i）未标注数据用于基于提示的少样本学习，以及（ii）多个提示，能够逆转这一趋势。我们进一步证明，增加少样本示例数量及模型规模可提升原始少样本方法的对抗鲁棒性。总体而言，本研究揭示了基于提示的少样本方法在自然语言理解任务中的对抗鲁棒性评估问题。