The rapid advancement in generative artificial intelligence have enabled the creation of 3D human faces (HFs) for applications including media production, virtual reality, security, healthcare, and game development, etc. However, assessing the quality and realism of these AI-generated 3D human faces remains a significant challenge due to the subjective nature of human perception and innate perceptual sensitivity to facial features. To this end, we conduct a comprehensive study on the quality assessment of AI-generated 3D human faces. We first introduce Gen3DHF, a large-scale benchmark comprising 2,000 videos of AI-Generated 3D Human Faces along with 4,000 Mean Opinion Scores (MOS) collected across two dimensions, i.e., quality and authenticity, 2,000 distortion-aware saliency maps and distortion descriptions. Based on Gen3DHF, we propose LMME3DHF, a Large Multimodal Model (LMM)-based metric for Evaluating 3DHF capable of quality and authenticity score prediction, distortion-aware visual question answering, and distortion-aware saliency prediction. Experimental results show that LMME3DHF achieves state-of-the-art performance, surpassing existing methods in both accurately predicting quality scores for AI-generated 3D human faces and effectively identifying distortion-aware salient regions and distortion types, while maintaining strong alignment with human perceptual judgments. Both the Gen3DHF database and the LMME3DHF will be released upon the publication.
翻译:生成式人工智能的快速发展使得创建3D人脸成为可能,其应用涵盖媒体制作、虚拟现实、安防、医疗保健和游戏开发等领域。然而,由于人类感知的主观性以及对面部特征固有的感知敏感性,评估这些AI生成的3D人脸的质量和真实感仍然是一个重大挑战。为此,我们对AI生成的3D人脸的质量评估进行了全面研究。我们首先提出了Gen3DHF,这是一个大规模基准数据集,包含2,000个AI生成的3D人脸视频、在质量和真实感两个维度上收集的4,000个平均意见得分、2,000张失真感知显著图以及失真描述。基于Gen3DHF,我们提出了LMME3DHF,一种基于大语言多模态模型的3D人脸评估指标,能够进行质量和真实感分数预测、失真感知视觉问答以及失真感知显著区域预测。实验结果表明,LMME3DHF在准确预测AI生成3D人脸的质量分数、有效识别失真感知显著区域和失真类型方面均达到了最先进的性能,超越了现有方法,同时与人类感知判断保持了高度一致性。Gen3DHF数据库和LMME3DHF模型将在论文发表时同步开源。