While state-of-the-art facial expression recognition (FER) classifiers achieve a high level of accuracy, they lack interpretability, an important aspect for end-users. To recognize basic facial expressions, experts resort to a codebook associating a set of spatial action units to a facial expression. In this paper, we follow the same expert footsteps, and propose a learning strategy that allows us to explicitly incorporate spatial action units (aus) cues into the classifier's training to build a deep interpretable model. In particular, using this aus codebook, input image expression label, and facial landmarks, a single action units heatmap is built to indicate the most discriminative regions of interest in the image w.r.t the facial expression. We leverage this valuable spatial cue to train a deep interpretable classifier for FER. This is achieved by constraining the spatial layer features of a classifier to be correlated with \aus map. Using a composite loss, the classifier is trained to correctly classify an image while yielding interpretable visual layer-wise attention correlated with aus maps, simulating the experts' decision process. This is achieved using only the image class expression as supervision and without any extra manual annotations. Moreover, our method is generic. It can be applied to any CNN- or transformer-based deep classifier without the need for architectural change or adding significant training time. Our extensive evaluation on two public benchmarks RAFDB, and AFFECTNET datasets shows that our proposed strategy can improve layer-wise interpretability without degrading classification performance. In addition, we explore a common type of interpretable classifiers that rely on Class-Activation Mapping methods (CAMs), and we show that our training technique improves the CAM interpretability.

翻译：虽然最先进的面部表情识别（FER）分类器能够达到高精度，但它们缺乏可解释性——这是最终用户关注的重要方面。为识别基本面部表情，专家们采用将一组空间动作单元与面部表情关联的编码本。本文遵循专家的思路，提出一种学习策略，通过显式地将空间动作单元（AUs）线索融入分类器训练，构建深度可解释模型。具体而言，利用该AU编码本、输入图像的表情标签及面部关键点，我们构建了单一动作单元热力图，以标示图像中与面部表情最相关的判别性区域。我们利用这一有价值的空间线索训练深度可解释FER分类器，通过约束分类器的空间层特征与AU热力图的相关性实现。采用复合损失函数，分类器在正确分类图像的同时，生成与AU热力图相关的逐层可视化注意力，模拟专家决策过程。该方法仅以图像类标签为监督信号，无需额外人工标注。此外，我们的方法具有通用性，可应用于任何基于CNN或Transformer的深度分类器，无需改变网络架构或显著增加训练时间。在RAFDB和AFFECTNET两个公开基准数据集上的广泛评估表明，所提策略可在不降低分类性能的前提下提升逐层可解释性。同时，我们探究了依赖类激活映射（CAM）方法的常见可解释分类器，证明我们的训练技术能改善CAM的可解释性。