Recently, deep convolution neural networks (CNNs) steered face super-resolution methods have achieved great progress in restoring degraded facial details by jointly training with facial priors. However, these methods have some obvious limitations. On the one hand, multi-task joint learning requires additional marking on the dataset, and the introduced prior network will significantly increase the computational cost of the model. On the other hand, the limited receptive field of CNN will reduce the fidelity and naturalness of the reconstructed facial images, resulting in suboptimal reconstructed images. In this work, we propose an efficient CNN-Transformer Cooperation Network (CTCNet) for face super-resolution tasks, which uses the multi-scale connected encoder-decoder architecture as the backbone. Specifically, we first devise a novel Local-Global Feature Cooperation Module (LGCM), which is composed of a Facial Structure Attention Unit (FSAU) and a Transformer block, to promote the consistency of local facial detail and global facial structure restoration simultaneously. Then, we design an efficient Feature Refinement Module (FRM) to enhance the encoded features. Finally, to further improve the restoration of fine facial details, we present a Multi-scale Feature Fusion Unit (MFFU) to adaptively fuse the features from different stages in the encoder procedure. Extensive evaluations on various datasets have assessed that the proposed CTCNet can outperform other state-of-the-art methods significantly. Source code will be available at https://github.com/IVIPLab/CTCNet.

翻译：近年来，基于深度卷积神经网络（CNN）的人脸超分辨率方法通过联合训练面部先验，在恢复降解的面部细节方面取得了显著进展。然而，这些方法存在一些明显局限：一方面，多任务联合学习需要对数据集进行额外标注，且引入的先验网络会显著增加模型的计算成本；另一方面，CNN有限的感受野会降低重建人脸图像的保真度和自然性，导致重建图像次优。本研究提出一种高效的人脸超分辨率CNN-Transformer协作网络（CTCNet），采用多尺度连接的编码器-解码器架构作为骨干网络。具体而言，我们首先设计了新颖的局部-全局特征协作模块（LGCM），该模块由面部结构注意力单元（FSAU）和Transformer块组成，以同时促进局部面部细节与全局面部结构恢复的一致性。其次，我们设计了一个高效的特征精炼模块（FRM）以增强编码特征。最后，为进一步改善精细面部细节的恢复，我们提出多尺度特征融合单元（MFFU），自适应融合编码器不同阶段的特征。在多个数据集上的广泛评估证实，所提出的CTCNet能显著优于其他最先进方法。源代码将于https://github.com/IVIPLab/CTCNet 提供。