Vision-and-Language models such as CLIP have demonstrated remarkable effectiveness across a wide range of tasks. However, these models are typically trained on web-scale data, which can introduce inappropriate content and lead to the development of unsafe and biased behavior. This, in turn, hampers their applicability in sensitive and trustworthy contexts and could raise significant concern in their adoption. To overcome these limitations, we introduce a methodology to make Vision-and-Language models safer by removing their sensitivity to not-safe-for-work concepts. We show how this can be done by distilling from a large language model which converts between safe and unsafe sentences and which is fine-tuned starting from just 100 manually-curated pairs. We conduct extensive experiments on the resulting embedding space for both retrieval and text-to-image generation, where we show that our model can also be properly employed with pre-trained image generators. Our source code and trained models are available at: https://github.com/aimagelab/safe-clip.

翻译：视觉-语言模型（如CLIP）已在众多任务中展现出卓越的性能。然而，这些模型通常基于网络规模数据进行训练，这可能导致其引入不当内容，并产生不安全及有偏见的行为。这种缺陷不仅限制了其在敏感和可信场景中的适用性，还可能引发对其部署的严重担忧。为克服这些局限，我们提出了一种方法，通过移除模型对不适宜工作场所（NSFW）概念的敏感性，使其更为安全。本文展示了如何利用大型语言模型蒸馏实现该目标：该模型可转换安全与不安全句子，且仅需从100个人工配对样本出发进行微调。我们在所得嵌入空间上针对检索和文本到图像生成任务开展了大量实验，结果表明我们的模型还可有效兼容预训练图像生成器。源代码与训练模型已开源至：https://github.com/aimagelab/safe-clip。