The performance of CLIP in dynamic facial expression recognition (DFER) task doesn't yield exceptional results as observed in other CLIP-based classification tasks. While CLIP's primary objective is to achieve alignment between images and text in the feature space, DFER poses challenges due to the abstract nature of text and the dynamic nature of video, making label representation limited and perfect alignment difficult. To address this issue, we have designed A$^{3}$lign-DFER, which introduces a new DFER labeling paradigm to comprehensively achieve alignment, thus enhancing CLIP's suitability for the DFER task. Specifically, our A$^{3}$lign-DFER method is designed with multiple modules that work together to obtain the most suitable expanded-dimensional embeddings for classification and to achieve alignment in three key aspects: affective, dynamic, and bidirectional. We replace the input label text with a learnable Multi-Dimensional Alignment Token (MAT), enabling alignment of text to facial expression video samples in both affective and dynamic dimensions. After CLIP feature extraction, we introduce the Joint Dynamic Alignment Synchronizer (JAS), further facilitating synchronization and alignment in the temporal dimension. Additionally, we implement a Bidirectional Alignment Training Paradigm (BAP) to ensure gradual and steady training of parameters for both modalities. Our insightful and concise A$^{3}$lign-DFER method achieves state-of-the-art results on multiple DFER datasets, including DFEW, FERV39k, and MAFW. Extensive ablation experiments and visualization studies demonstrate the effectiveness of A$^{3}$lign-DFER. The code will be available in the future.

翻译：在动态面部表情识别（DFER）任务中，CLIP的性能并未如其他基于CLIP的分类任务那般取得卓越效果。尽管CLIP的主要目标是在特征空间中实现图像与文本的对齐，但DFER因文本的抽象性和视频的动态性而面临挑战，导致标签表征受限且难以实现完美对齐。为解决此问题，我们设计了A$^{3}$lign-DFER，引入一种新的DFER标签范式以全面实现对齐，从而增强CLIP在DFER任务中的适用性。具体而言，我们的A$^{3}$lign-DFER方法通过多个模块协同工作，获取最适合分类的扩展维度嵌入，并在情感、动态和双向三个关键维度实现对齐。我们用可学习的多维对齐标记（MAT）替换输入标签文本，使文本在情感和动态维度上均能与面部表情视频样本对齐。在CLIP特征提取后，我们引入联合动态对齐同步器（JAS），进一步促进时间维度上的同步与对齐。此外，我们实现了双向对齐训练范式（BAP），以确保两种模态的参数在训练过程中逐步且稳定地适配。我们精炼且富有洞见的A$^{3}$lign-DFER方法在多个DFER数据集（包括DFEW、FERV39k和MAFW）上取得了最先进的结果。广泛的消融实验和可视化研究证明了A$^{3}$lign-DFER的有效性。代码将在未来公开。