Interpretation and visualization of the behavior of detection transformers tends to highlight the locations in the image that the model attends to, but it provides limited insight into the \emph{semantics} that the model is focusing on. This paper introduces an extension to detection transformers that constructs prototypical local features and uses them in object detection. These custom features, which we call prototypical parts, are designed to be mutually exclusive and align with the classifications of the model. The proposed extension consists of a bottleneck module, the prototype neck, that computes a discretized representation of prototype activations and a new loss term that matches prototypes to object classes. This setup leads to interpretable representations in the prototype neck, allowing visual inspection of the image content perceived by the model and a better understanding of the model's reliability. We show experimentally that our method incurs only a limited performance penalty, and we provide examples that demonstrate the quality of the explanations provided by our method, which we argue outweighs the performance penalty.

翻译：检测转换器行为解释与可视化通常着重于模型关注的图像位置，但对模型聚焦的语义信息揭示有限。本文提出检测转换器的扩展方法，构建原型局部特征并将其用于目标检测。这些定制特征称为原型部件，设计为互斥且与模型分类对齐。该扩展包含瓶颈模块（原型颈部），用于计算原型激活的离散化表示，以及将原型匹配至物体类别的新损失项。这种结构在原型颈部产生可解释表示，支持对模型感知图像内容的视觉审查，并加深对模型可靠性的理解。实验表明，我们的方法仅带来有限的性能损失，同时提供的解释质量优于这种损失，我们通过实例证明了这一点。