Semi-supervised learning has emerged as a promising approach to tackle the challenge of label scarcity in facial expression recognition (FER) task. However, current state-of-the-art methods primarily focus on one side of the coin, i.e., generating high-quality pseudo-labels, while overlooking the other side: enhancing expression-relevant representations. In this paper, we unveil both sides of the coin by proposing a unified framework termed hierarchicaL dEcoupling And Fusing (LEAF) to coordinate expression-relevant representations and pseudo-labels for semi-supervised FER. LEAF introduces a hierarchical expression-aware aggregation strategy that operates at three levels: semantic, instance, and category. (1) At the semantic and instance levels, LEAF decouples representations into expression-agnostic and expression-relevant components, and adaptively fuses them using learnable gating weights. (2) At the category level, LEAF assigns ambiguous pseudo-labels by decoupling predictions into positive and negative parts, and employs a consistency loss to ensure agreement between two augmented views of the same image. Extensive experiments on benchmark datasets demonstrate that by unveiling and harmonizing both sides of the coin, LEAF outperforms state-of-the-art semi-supervised FER methods, effectively leveraging both labeled and unlabeled data. Moreover, the proposed expression-aware aggregation strategy can be seamlessly integrated into existing semi-supervised frameworks, leading to significant performance gains. Our code is available at https://anonymous.4open.science/r/LEAF-BC57/.

翻译：半监督学习已成为应对面部表情识别（FER）任务中标签稀缺问题的一种有前景的方法。然而，当前最先进的技术主要关注硬币的一面，即生成高质量伪标签，却忽视了另一面：增强与表情相关的表示。本文通过提出一个名为分层解耦与融合（LEAF）的统一框架，揭示了硬币的两面，以协调半监督FER中的表情相关表示和伪标签。LEAF引入了一种分层的表情感知聚合策略，在语义、实例和类别三个层面运作：(1) 在语义和实例层面，LEAF将表示解耦为表情无关和表情相关成分，并通过可学习的门控权重自适应地融合它们；(2) 在类别层面，LEAF通过将预测解耦为正负部分来分配模糊伪标签，并采用一致性损失确保同一图像两个增强视图之间的一致性。在基准数据集上的大量实验表明，通过揭示并协调硬币的两面，LEAF超越了最先进的半监督FER方法，有效利用了有标签和无标签数据。此外，所提出的表情感知聚合策略可无缝集成到现有半监督框架中，带来显著的性能提升。我们的代码可在https://anonymous.4open.science/r/LEAF-BC57/获取。