Facial expression recognition (FER) algorithms work well in constrained environments with little or no occlusion of the face. However, real-world face occlusion is prevalent, most notably with the need to use a face mask in the current Covid-19 scenario. While there are works on the problem of occlusion in FER, little has been done before on the particular face mask scenario. Moreover, the few works in this area largely use synthetically created masked FER datasets. Motivated by these challenges posed by the pandemic to FER, we present a novel dataset, the Masked Student Dataset of Expressions or MSD-E, consisting of 1,960 real-world non-masked and masked facial expression images collected from 142 individuals. Along with the issue of obfuscated facial features, we illustrate how other subtler issues in masked FER are represented in our dataset. We then provide baseline results using ResNet-18, finding that its performance dips in the non-masked case when trained for FER in the presence of masks. To tackle this, we test two training paradigms: contrastive learning and knowledge distillation, and find that they increase the model's performance in the masked scenario while maintaining its non-masked performance. We further visualise our results using t-SNE plots and Grad-CAM, demonstrating that these paradigms capitalise on the limited features available in the masked scenario. Finally, we benchmark SOTA methods on MSD-E.

翻译：面部表情识别（FER）算法在面部无遮挡或少量遮挡的受控环境下表现良好。然而，真实场景中面部遮挡普遍存在，尤其在当前新冠疫情背景下佩戴口罩的需求尤为突出。尽管已有研究关注FER中的遮挡问题，但针对口罩这一特定遮挡场景的工作仍十分有限。现有少量研究主要使用合成生成的口罩FER数据集。受疫情对FER带来的挑战启发，我们提出一个新型数据集——掩膜学生表情数据集（MSD-E），该数据集包含来自142名个体的1,960张真实场景下未佩戴口罩和佩戴口罩的面部表情图像。除了面部特征模糊问题外，我们还展示了掩膜FER中其他细微问题在数据集中的体现。我们使用ResNet-18提供基线结果，发现在训练涉及口罩的FER模型时，其未佩戴口罩场景的性能出现下降。为解决该问题，我们测试了两种训练范式：对比学习和知识蒸馏。实验表明，这两种方法在提升模型口罩场景性能的同时，维持了其在未佩戴口罩场景的表现。我们进一步通过t-SNE图和Grad-CAM进行可视化分析，证明这些范式有效利用了口罩场景下有限的可用特征。最后，我们在MSD-E上对当前最优方法进行了基准测试。