Dynamic facial expression recognition (DFER) is essential to the development of intelligent and empathetic machines. Prior efforts in this field mainly fall into supervised learning paradigm, which is restricted by the limited labeled data in existing datasets. Inspired by recent unprecedented success of masked autoencoders (e.g., VideoMAE), this paper proposes MAE-DFER, a novel self-supervised method which leverages large-scale self-supervised pre-training on abundant unlabeled data to advance the development of DFER. Since the vanilla Vision Transformer (ViT) employed in VideoMAE requires substantial computation during fine-tuning, MAE-DFER develops an efficient local-global interaction Transformer (LGI-Former) as the encoder. LGI-Former first constrains self-attention in local spatiotemporal regions and then utilizes a small set of learnable representative tokens to achieve efficient local-global information exchange, thus avoiding the expensive computation of global space-time self-attention in ViT. Moreover, in addition to the standalone appearance content reconstruction in VideoMAE, MAE-DFER also introduces explicit facial motion modeling to encourage LGI-Former to excavate both static appearance and dynamic motion information. Extensive experiments on six datasets show that MAE-DFER consistently outperforms state-of-the-art supervised methods by significant margins, verifying that it can learn powerful dynamic facial representations via large-scale self-supervised pre-training. Besides, it has comparable or even better performance than VideoMAE, while largely reducing the computational cost (about 38\% FLOPs). We believe MAE-DFER has paved a new way for the advancement of DFER and can inspire more relavant research in this field and even other related tasks. Codes and models are publicly available at https://github.com/sunlicai/MAE-DFER.

翻译：动态面部表情识别（DFER）对发展智能且富有同理心的机器至关重要。该领域先前研究主要采用监督学习范式，但受限于现有数据集中有限的标注数据。受掩码自编码器（如VideoMAE）近期空前成功的启发，本文提出MAE-DFER——一种新颖的自监督方法，通过在大规模无标注数据上进行自监督预训练，推动DFER领域发展。由于VideoMAE中采用的原始视觉Transformer（ViT）在微调时需要大量计算，MAE-DFER开发了高效的局部-全局交互Transformer（LGI-Former）作为编码器。LGI-Former首先将自注意力约束在局部时空区域，随后利用少量可学习表征令牌实现高效的局部-全局信息交换，从而避免了ViT中全局时空自注意力带来的高昂计算成本。此外，除VideoMAE中独立的外观内容重建外，MAE-DFER还引入显式的面部运动建模，促使LGI-Former同时挖掘静态外观与动态运动信息。在六个数据集上的大量实验表明，MAE-DFER以显著优势持续超越最先进的监督方法，验证了其通过大规模自监督预训练能够学习到强大的动态面部表征。同时，其性能与VideoMAE相当甚至更优，并将计算成本大幅降低约38%的FLOPs。我们相信MAE-DFER为DFER的发展开辟了新路径，并能为该领域及其他相关任务的研究提供启发。代码与模型已公开于https://github.com/sunlicai/MAE-DFER。