Although state-of-the-art classifiers for facial expression recognition (FER) can achieve a high level of accuracy, they lack interpretability, an important feature for end-users. Experts typically associate spatial action units (\aus) from a codebook to facial regions for the visual interpretation of expressions. In this paper, the same expert steps are followed. A new learning strategy is proposed to explicitly incorporate \au cues into classifier training, allowing to train deep interpretable models. During training, this \au codebook is used, along with the input image expression label, and facial landmarks, to construct a \au heatmap that indicates the most discriminative image regions of interest w.r.t the facial expression. This valuable spatial cue is leveraged to train a deep interpretable classifier for FER. This is achieved by constraining the spatial layer features of a classifier to be correlated with \au heatmaps. Using a composite loss, the classifier is trained to correctly classify an image while yielding interpretable visual layer-wise attention correlated with \au maps, simulating the expert decision process. Our strategy only relies on image class expression for supervision, without additional manual annotations. Our new strategy is generic, and can be applied to any deep CNN- or transformer-based classifier without requiring any architectural change or significant additional training time. Our extensive evaluation on two public benchmarks \rafdb, and \affectnet datasets shows that our proposed strategy can improve layer-wise interpretability without degrading classification performance. In addition, we explore a common type of interpretable classifiers that rely on class activation mapping (CAM) methods, and show that our approach can also improve CAM interpretability.

翻译：尽管最先进的面部表情识别（FER）分类器能够实现高精度，但它们缺乏可解释性——这是终端用户所需的重要特性。专家通常将编码本中的空间动作单元（AUs）与面部区域关联，以实现表情的视觉解释。本文遵循相同的专家步骤，提出一种新的学习策略，将AU线索显式融入分类器训练中，从而训练出深度可解释模型。训练过程中，利用AU编码本、输入图像的表情标签及面部关键点，构建指示与面部表情最具判别性的图像感兴趣区域的AU热图。这一宝贵的空间线索被用于训练深度可解释的FER分类器，具体通过约束分类器空间层特征与AU热图的相关性实现。采用复合损失函数，分类器在正确分类图像的同时，生成与AU图相关的可解释逐层视觉注意力，模拟专家决策过程。本策略仅依赖图像类表情标签进行监督，无需额外人工标注。该新策略具有通用性，可应用于任何基于深度CNN或Transformer的分类器，无需修改架构或显著增加训练时间。在RAF-DB和AffectNet两个公开基准数据集上的广泛评估表明，所提策略能在不降低分类性能的前提下提升逐层可解释性。此外，本文探索了依赖类激活映射（CAM）方法的常见可解释分类器，并证明我们的方法也能提升CAM的可解释性。