Zero-Shot Learning (ZSL) focuses on classifying samples of unseen classes with only their side semantic information presented during training. It cannot handle real-life, open-world scenarios where there are test samples of unknown classes for which neither samples (e.g., images) nor their side semantic information is known during training. Open-Set Recognition (OSR) is dedicated to addressing the unknown class issue, but existing OSR methods are not designed to model the semantic information of the unseen classes. To tackle this combined ZSL and OSR problem, we consider the case of "Zero-Shot Open-Set Recognition" (ZS-OSR), where a model is trained under the ZSL setting but it is required to accurately classify samples from the unseen classes while being able to reject samples from the unknown classes during inference. We perform large experiments on combining existing state-of-the-art ZSL and OSR models for the ZS-OSR task on four widely used datasets adapted from the ZSL task, and reveal that ZS-OSR is a non-trivial task as the simply combined solutions perform badly in distinguishing the unseen-class and unknown-class samples. We further introduce a novel approach specifically designed for ZS-OSR, in which our model learns to generate adversarial semantic embeddings of the unknown classes to train an unknowns-informed ZS-OSR classifier. Extensive empirical results show that our method 1) substantially outperforms the combined solutions in detecting the unknown classes while retaining the classification accuracy on the unseen classes and 2) achieves similar superiority under generalized ZS-OSR settings.

翻译：零样本学习（ZSL）专注于利用训练期间仅提供的侧语义信息对未见类别的样本进行分类。然而，它无法处理真实世界的开放场景，其中存在未知类别的测试样本，这些样本在训练期间既无样本（如图像）也无其侧语义信息。开放集识别（OSR）致力于解决未知类别问题，但现有OSR方法并非为建模未见类别的语义信息而设计。为应对这一结合ZSL与OSR的问题，我们考虑“零样本开放集识别”（ZS-OSR）情形，即模型在ZSL设置下训练，但需在推理时准确对未见类别样本进行分类，同时能够拒识未知类别样本。我们通过将现有最先进的ZSL与OSR模型结合，在基于ZSL任务改编的四个广泛使用的数据集上针对ZS-OSR任务进行了大规模实验，揭示出ZS-OSR并非简单任务——直接组合的方案在区分未见类别与未知类别样本时表现不佳。我们进一步引入一种专为ZS-OSR设计的新方法，其中模型学习生成未知类别的对抗性语义嵌入，以训练信息感知的ZS-OSR分类器。大量实证结果表明，我们的方法：1）在保留未见类别分类精度的同时，在检测未知类别方面显著优于组合方案；2）在广义ZS-OSR设置下同样表现出类似优势。