Many classification problems consider classes that form a hierarchy. Classifiers that are aware of this hierarchy may be able to make confident predictions at a coarse level despite being uncertain at the fine-grained level. While it is generally possible to vary the granularity of predictions using a threshold at inference time, most contemporary work considers only leaf-node prediction, and almost no prior work has compared methods at multiple operating points. We present an efficient algorithm to produce operating characteristic curves for any method that assigns a score to every class in the hierarchy. Applying this technique to evaluate existing methods reveals that top-down classifiers are dominated by a naive flat softmax classifier across the entire operating range. We further propose two novel loss functions and show that a soft variant of the structured hinge loss is able to significantly outperform the flat baseline. Finally, we investigate the poor accuracy of top-down classifiers and demonstrate that they perform relatively well on unseen classes. Code is available online at https://github.com/jvlmdr/hiercls.

翻译：许多分类问题涉及构成层级结构的类别。能够感知这种层级的分类器，即使对细粒度类别不确定，也能在粗粒度层面做出可靠预测。虽然通常可以在推理时通过阈值调整预测的粒度，但当代研究大多仅考虑叶节点预测，且几乎没有先前工作对多操作点下的方法进行比较。我们提出一种高效算法，可为层级中每个类别分配得分的任意方法生成操作特征曲线。将此技术应用于评估现有方法时发现，整个操作范围内，自顶向下分类器的性能均被朴素扁平softmax分类器所主导。我们进一步提出两种新颖的损失函数，并证明结构化铰链损失的软变体能够显著超越扁平基线。最后，我们探究自顶向下分类器的低准确率问题，并证明它们在未见类别上表现相对较好。代码可在 https://github.com/jvlmdr/hiercls 获取。