Open vocabulary models (e.g. CLIP) have shown strong performance on zero-shot classification through their ability generate embeddings for each class based on their (natural language) names. Prior work has focused on improving the accuracy of these models through prompt engineering or by incorporating a small amount of labeled downstream data (via finetuning). However, there has been little focus on improving the richness of the class names themselves, which can pose issues when class labels are coarsely-defined and are uninformative. We propose Classification with Hierarchical Label Sets (or CHiLS), an alternative strategy for zero-shot classification specifically designed for datasets with implicit semantic hierarchies. CHiLS proceeds in three steps: (i) for each class, produce a set of subclasses, using either existing label hierarchies or by querying GPT-3; (ii) perform the standard zero-shot CLIP procedure as though these subclasses were the labels of interest; (iii) map the predicted subclass back to its parent to produce the final prediction. Across numerous datasets with underlying hierarchical structure, CHiLS leads to improved accuracy in situations both with and without ground-truth hierarchical information. CHiLS is simple to implement within existing zero-shot pipelines and requires no additional training cost. Code is available at: https://github.com/acmi-lab/CHILS.

翻译：开放词汇模型（如CLIP）通过根据类别（自然语言）名称生成嵌入向量，在零样本分类中展现出强大性能。已有研究聚焦于通过提示工程或结合少量带标签下游数据（通过微调）提升这些模型的准确率，但鲜有关注类别名称本身丰富度的改进——当类别标签定义粗粒度且缺乏信息量时，这会引发问题。本文提出基于层级标签集的分类方法（简称CHiLS），这是一种专为具有隐式语义层级结构的数据集设计的零样本分类替代策略。CHiLS包含三个步骤：（i）利用现有标签层级或查询GPT-3，为每个类别生成子类别集合；（ii）将这些子类别作为目标标签，执行标准零样本CLIP流程；（iii）将预测的子类别映射回其父类别，得到最终预测结果。在多个具有层级结构的数据集上，无论是否使用真实层级信息，CHiLS均能提升分类准确率。该方法易于集成至现有零样本流程，且无需额外训练成本。代码开源地址：https://github.com/acmi-lab/CHILS。