Zero-shot learning models achieve remarkable results on image classification for samples from classes that were not seen during training. However, such models must be trained from scratch with specialised methods: therefore, access to a training dataset is required when the need for zero-shot classification arises. In this paper, we aim to equip pre-trained models with zero-shot classification capabilities without the use of image data. We achieve this with our proposed Image-free Classifier Injection with Semantics (ICIS) that injects classifiers for new, unseen classes into pre-trained classification models in a post-hoc fashion without relying on image data. Instead, the existing classifier weights and simple class-wise descriptors, such as class names or attributes, are used. ICIS has two encoder-decoder networks that learn to reconstruct classifier weights from descriptors (and vice versa), exploiting (cross-)reconstruction and cosine losses to regularise the decoding process. Notably, ICIS can be cheaply trained and applied directly on top of pre-trained classification models. Experiments on benchmark ZSL datasets show that ICIS produces unseen classifier weights that achieve strong (generalised) zero-shot classification performance. Code is available at https://github.com/ExplainableML/ImageFreeZSL .

翻译：零样本学习模型在对训练期间未见过的类别样本进行图像分类时取得了显著成果。然而，此类模型必须使用专门方法从头训练：因此，当需要零样本分类时，必须访问训练数据集。本文旨在无需使用图像数据的情况下，为预训练模型赋予零样本分类能力。我们通过提出的无图像语义分类器注入（ICIS）方法实现该目标，该方法以事后方式将新类别的分类器注入预训练分类模型，而无需依赖图像数据。相反，该方法使用现有分类器权重和简单的类别描述符（如类别名称或属性）。ICIS包含两个编码器-解码器网络，它们学习从描述符重建分类器权重（反之亦然），利用（交叉）重建损失和余弦损失来规范化解码过程。值得注意的是，ICIS可廉价训练并直接应用于预训练分类模型之上。在基准ZSL数据集上的实验表明，ICIS生成的未见类别分类器权重实现了强大的（广义）零样本分类性能。代码开源在 https://github.com/ExplainableML/ImageFreeZSL 。