Recently, deep convolution neural networks (CNNs) steered face super-resolution methods have achieved great progress in restoring degraded facial details by jointly training with facial priors. However, these methods have some obvious limitations. On the one hand, multi-task joint learning requires additional marking on the dataset, and the introduced prior network will significantly increase the computational cost of the model. On the other hand, the limited receptive field of CNN will reduce the fidelity and naturalness of the reconstructed facial images, resulting in suboptimal reconstructed images. In this work, we propose an efficient CNN-Transformer Cooperation Network (CTCNet) for face super-resolution tasks, which uses the multi-scale connected encoder-decoder architecture as the backbone. Specifically, we first devise a novel Local-Global Feature Cooperation Module (LGCM), which is composed of a Facial Structure Attention Unit (FSAU) and a Transformer block, to promote the consistency of local facial detail and global facial structure restoration simultaneously. Then, we design an efficient Feature Refinement Module (FRM) to enhance the encoded features. Finally, to further improve the restoration of fine facial details, we present a Multi-scale Feature Fusion Unit (MFFU) to adaptively fuse the features from different stages in the encoder procedure. Extensive evaluations on various datasets have assessed that the proposed CTCNet can outperform other state-of-the-art methods significantly. Source code will be available at https://github.com/IVIPLab/CTCNet.

翻译：近年来，基于深度卷积神经网络（CNN）的人脸超分辨率方法通过联合训练面部先验，在恢复退化面部细节方面取得了显著进展。然而，这些方法存在明显局限：一方面，多任务联合学习需要对数据集进行额外标注，且引入的先验网络会显著增加模型计算成本；另一方面，CNN有限的感受野会降低重建面部图像的逼真度与自然度，导致重建结果欠优。本文提出一种高效的CNN-Transformer协作网络（CTCNet）用于人脸超分辨率任务，采用多尺度连接的编码器-解码器架构作为主干网络。具体而言，我们首先设计了一种新颖的局部-全局特征协作模块（LGCM），该模块由面部结构注意力单元（FSAU）和Transformer块组成，以同时促进局部面部细节与全局面部结构恢复的一致性。其次，我们设计了一种高效的特征精炼模块（FRM）以增强编码特征。最后，为进一步提升精细面部细节的恢复效果，我们提出了一种多尺度特征融合单元（MFFU），自适应融合编码器不同阶段提取的特征。多数据集上的广泛评估表明，所提出的CTCNet能够显著超越其他最先进方法。源代码将在https://github.com/IVIPLab/CTCNet 公开。