As computer vision continues to advance and finds widespread applications across various domains, the need for interpretability in deep learning models becomes paramount. Existing methods often resort to post-hoc techniques or prototypes to explain the decision-making process, which can be indirect and lack intrinsic illustration. In this research, we introduce ViTree, a novel approach for fine-grained visual categorization that combines the popular vision transformer as a feature extraction backbone with neural decision trees. By traversing the tree paths, ViTree effectively selects patches from transformer-processed features to highlight informative local regions, thereby refining representations in a step-wise manner. Unlike previous tree-based models that rely on soft distributions or ensembles of paths, ViTree selects a single tree path, offering a clearer and simpler decision-making process. This patch and path selectivity enhances model interpretability of ViTree, enabling better insights into the model's inner workings. Remarkably, extensive experimentation validates that this streamlined approach surpasses various strong competitors and achieves state-of-the-art performance while maintaining exceptional interpretability which is proved by multi-perspective methods. Code can be found at https://github.com/SJTU-DeepVisionLab/ViTree.

翻译：随着计算机视觉的持续发展及其在各领域的广泛应用，深度学习模型的可解释性变得至关重要。现有方法常采用事后解释或原型来解释决策过程，这些方法可能较为间接且缺乏内在说明。本研究提出ViTree，一种结合了流行视觉变换器（作为特征提取主干）与神经决策树的细粒度视觉分级新方法。通过遍历树路径，ViTree有效从变换器处理的特征中选择图像块以突出信息丰富的局部区域，从而逐步优化表征。与以往依赖软分布或路径集成的树模型不同，ViTree选择单条树路径，提供更清晰简洁的决策过程。这种图像块与路径的选择性增强了ViTree的模型可解释性，使人们能更深入地理解模型内部运作机制。值得注意的是，大量实验证明，这种简化方法超越了多种强基线方法，达到最先进性能，同时保持了通过多视角方法证实的卓越可解释性。代码见https://github.com/SJTU-DeepVisionLab/ViTree。