In realistic open-set scenarios where labels of a part of testing data are totally unknown, current prompt methods on vision-language (VL) models always predict the unknown classes as the downstream training classes. The exhibited label bias causes difficulty in the open set recognition (OSR), by which an image should be correctly predicted as one of the known classes or the unknown one. To learn prompts in open-set scenarios, we propose the Regularized prompt Tuning (R-Tuning) to mitigate the label bias. It introduces open words from the WordNet to extend the range of words forming the prompt texts from only closed-set label words to more. Thus, prompts are tuned in a simulated open-set scenario. Besides, inspired by the observation that classifying directly on large datasets causes a much higher false positive rate than on small datasets, we propose the Combinatorial Tuning and Testing (CTT) strategy for improving performance. CTT decomposes R-Tuning on large datasets as multiple independent group-wise tuning on fewer classes, then makes comprehensive predictions by selecting the optimal sub-prompt. For fair comparisons, we construct new baselines for OSR based on VL models, especially for prompt methods. Our method achieves the best results on datasets with various scales. Extensive ablation studies validate the effectiveness of our method.

翻译：在现实开放场景中，部分测试数据的标签完全未知，当前基于视觉-语言（VL）模型的提示方法总是将未知类别预测为下游训练类别。这种标签偏差导致开放集识别（OSR）困难，即图像应被正确预测为已知类别或未知类别。为了在开放场景中学习提示，我们提出正则化提示微调（R-Tuning）以缓解标签偏差。该方法引入WordNet中的开放词汇，将构成提示文本的词汇范围从仅包含封闭集标签词汇扩展至更多词汇。由此，提示在模拟的开放场景中进行微调。此外，受直接在大数据集上分类比在小数据集上分类产生更高假阳性率的观察启发，我们提出组合微调与测试（CTT）策略以提升性能。CTT将大数据集上的R-Tuning分解为多个独立的逐组微调（每组类别数较少），然后通过选择最优子提示进行综合预测。为公平比较，我们基于VL模型构建了OSR的新基线，尤其针对提示方法。我们的方法在不同规模的数据集上均取得了最佳结果。大量消融研究验证了本方法的有效性。