Zero-shot inference is a powerful paradigm that enables the use of large pretrained models for downstream classification tasks without further training. However, these models are vulnerable to inherited biases that can impact their performance. The traditional solution is fine-tuning, but this undermines the key advantage of pretrained models, which is their ability to be used out-of-the-box. We propose RoboShot, a method that improves the robustness of pretrained model embeddings in a fully zero-shot fashion. First, we use zero-shot language models (LMs) to obtain useful insights from task descriptions. These insights are embedded and used to remove harmful and boost useful components in embeddings -- without any supervision. Theoretically, we provide a simple and tractable model for biases in zero-shot embeddings and give a result characterizing under what conditions our approach can boost performance. Empirically, we evaluate RoboShot on nine image and NLP classification tasks and show an average improvement of 15.98% over several zero-shot baselines. Additionally, we demonstrate that RoboShot is compatible with a variety of pretrained and language models.

翻译：零样本推理是一种强大的范式，使得大型预训练模型能够在不经过额外训练的情况下直接用于下游分类任务。然而，这些模型易受继承性偏差的影响，从而可能损害其性能。传统解决方案是通过微调，但这削弱了预训练模型的核心优势——即开箱即用的能力。我们提出RoboShot方法，以完全零样本的方式提升预训练模型嵌入的鲁棒性。首先，我们利用零样本语言模型从任务描述中获取有效洞察，将这些洞察嵌入后用于消除嵌入中有害成分并增强有益成分——全程无需任何监督。在理论上，我们为零样本嵌入中的偏差建立了简洁可处理的模型，并给出了表征何种条件下该方法能提升性能的理论结果。实验方面，我们在九个图像与NLP分类任务上评估RoboShot，显示其相较多种零样本基线方法平均提升15.98%。此外，我们证明RoboShot与多种预训练模型及语言模型均兼容。