Deep Neural Networks use thousands of mostly incomprehensible features to identify a single class, a decision no human can follow. We propose an interpretable sparse and low dimensional final decision layer in a deep neural network with measurable aspects of interpretability and demonstrate it on fine-grained image classification. We argue that a human can only understand the decision of a machine learning model, if the features are interpretable and only very few of them are used for a single decision. For that matter, the final layer has to be sparse and, to make interpreting the features feasible, low dimensional. We call a model with a Sparse Low-Dimensional Decision SLDD-Model. We show that a SLDD-Model is easier to interpret locally and globally than a dense high-dimensional decision layer while being able to maintain competitive accuracy. Additionally, we propose a loss function that improves a model's feature diversity and accuracy. Our more interpretable SLDD-Model only uses 5 out of just 50 features per class, while maintaining 97% to 100% of the accuracy on four common benchmark datasets compared to the baseline model with 2048 features.

翻译：摘要：深度神经网络通常使用数千个难以理解的特征来识别单个类别，这一决策过程人类无法追踪。我们提出了一种可解释的稀疏低维最终决策层，该层嵌入深度神经网络中，具备可量化的可解释性特征，并将其应用于细粒度图像分类。我们认为，只有当特征可解释且单个决策仅使用极少数特征时，人类才能理解机器学习模型的决策。为此，最终决策层必须稀疏且维度较低，以便能够有效解释这些特征。我们将这种具有稀疏低维决策层的模型称为SLDD-Model。研究表明，与密集高维决策层相比，SLDD-Model在局部和全局解释性方面更具优势，同时能保持具有竞争力的精度。此外，我们提出了一种可提升模型特征多样性与精度的损失函数。我们更具可解释性的SLDD-Model每个类别仅使用50个特征中的5个，而在四个通用基准数据集上，与使用2048个特征的基线模型相比，其精度保持在97%至100%之间。