Blind face restoration (BFR) on images has significantly progressed over the last several years, while real-world video face restoration (VFR), which is more challenging for more complex face motions such as moving gaze directions and facial orientations involved, remains unsolved. Typical BFR methods are evaluated on privately synthesized datasets or self-collected real-world low-quality face images, which are limited in their coverage of real-world video frames. In this work, we introduced new real-world datasets named FOS with a taxonomy of "Full, Occluded, and Side" faces from mainly video frames to study the applicability of current methods on videos. Compared with existing test datasets, FOS datasets cover more diverse degradations and involve face samples from more complex scenarios, which helps to revisit current face restoration approaches more comprehensively. Given the established datasets, we benchmarked both the state-of-the-art BFR methods and the video super resolution (VSR) methods to comprehensively study current approaches, identifying their potential and limitations in VFR tasks. In addition, we studied the effectiveness of the commonly used image quality assessment (IQA) metrics and face IQA (FIQA) metrics by leveraging a subjective user study. With extensive experimental results and detailed analysis provided, we gained insights from the successes and failures of both current BFR and VSR methods. These results also pose challenges to current face restoration approaches, which we hope stimulate future advances in VFR research.

翻译：盲人脸修复（BFR）在图像领域近年来取得了显著进展，但真实世界视频人脸修复（VFR）因涉及更复杂的面部运动（如移动的视线方向和面部朝向）而更具挑战性，至今仍未得到解决。典型的BFR方法通常在私有合成数据集或自收集的真实世界低质量人脸图像上进行评估，这些数据集对真实世界视频帧的覆盖有限。本研究引入了一套名为FOS的真实世界新数据集，该数据集基于“完整、遮挡和侧面”人脸的分类体系，主要来自视频帧，旨在评估现有方法在视频中的适用性。与现有测试数据集相比，FOS数据集覆盖了更多样化的退化类型，并包含来自更复杂场景的人脸样本，有助于更全面地审视当前的人脸修复方法。基于所建立的数据集，我们对最先进的BFR方法和视频超分辨率（VSR）方法进行了基准测试，以全面研究当前方法，并识别其在VFR任务中的潜力和局限性。此外，我们通过主观用户研究，评估了常用图像质量评估（IQA）指标和人脸IQA（FIQA）指标的有效性。结合大量实验结果与详细分析，我们从当前BFR和VSR方法的成功与失败中获得了深刻见解。这些结果也对当前人脸修复方法提出了挑战，我们期望能推动未来VFR研究的进步。