This paper explores a hierarchical prompting mechanism for the hierarchical image classification (HIC) task. Different from prior HIC methods, our hierarchical prompting is the first to explicitly inject ancestor-class information as a tokenized hint that benefits the descendant-class discrimination. We think it well imitates human visual recognition, i.e., humans may use the ancestor class as a prompt to draw focus on the subtle differences among descendant classes. We model this prompting mechanism into a Transformer with Hierarchical Prompting (TransHP). TransHP consists of three steps: 1) learning a set of prompt tokens to represent the coarse (ancestor) classes, 2) on-the-fly predicting the coarse class of the input image at an intermediate block, and 3) injecting the prompt token of the predicted coarse class into the intermediate feature. Though the parameters of TransHP maintain the same for all input images, the injected coarse-class prompt conditions (modifies) the subsequent feature extraction and encourages a dynamic focus on relatively subtle differences among the descendant classes. Extensive experiments show that TransHP improves image classification on accuracy (e.g., improving ViT-B/16 by +2.83% ImageNet classification accuracy), training data efficiency (e.g., +12.69% improvement under 10% ImageNet training data), and model explainability. Moreover, TransHP also performs favorably against prior HIC methods, showing that TransHP well exploits the hierarchical information.

翻译：本文探索了一种用于层次图像分类（HIC）任务的层次提示机制。与以往的HIC方法不同，我们的层次提示首次将祖先类信息显式注入为分词化提示，以提升后代类判别能力。我们认为这很好地模拟了人类视觉识别过程——即人类可能将祖先类作为提示，从而聚焦于后代类间的细微差异。我们将此提示机制建模为具有层次提示能力的Transformer（TransHP）。TransHP包含三个步骤：1）学习一组提示标记以表征粗粒度（祖先）类；2）在中间网络块中实时预测输入图像的粗粒度类；3）将预测粗粒度类的提示标记注入中间特征。虽然TransHP的参数对所有输入图像保持一致，但注入的粗粒度类提示会调节（修改）后续特征提取过程，并促使网络动态关注后代类间的相对细微差异。大量实验表明，TransHP在分类精度（如ViT-B/16在ImageNet上提升+2.83%）、训练数据效率（如仅用10% ImageNet训练数据时提升+12.69%）及模型可解释性方面均有效提升。此外，TransHP在分层信息利用方面也优于现有HIC方法，证明其能充分挖掘层次结构信息。