The visual quality of point clouds has been greatly emphasized since the ever-increasing 3D vision applications are expected to provide cost-effective and high-quality experiences for users. Looking back on the development of point cloud quality assessment (PCQA) methods, the visual quality is usually evaluated by utilizing single-modal information, i.e., either extracted from the 2D projections or 3D point cloud. The 2D projections contain rich texture and semantic information but are highly dependent on viewpoints, while the 3D point clouds are more sensitive to geometry distortions and invariant to viewpoints. Therefore, to leverage the advantages of both point cloud and projected image modalities, we propose a novel no-reference point cloud quality assessment (NR-PCQA) metric in a multi-modal fashion. In specific, we split the point clouds into sub-models to represent local geometry distortions such as point shift and down-sampling. Then we render the point clouds into 2D image projections for texture feature extraction. To achieve the goals, the sub-models and projected images are encoded with point-based and image-based neural networks. Finally, symmetric cross-modal attention is employed to fuse multi-modal quality-aware information. Experimental results show that our approach outperforms all compared state-of-the-art methods and is far ahead of previous NR-PCQA methods, which highlights the effectiveness of the proposed method. The code is available at https://github.com/zzc-1998/MM-PCQA.

翻译：随着日益增长的3D视觉应用期望为用户提供高性价比且高质量的体验，点云的视觉质量受到了极大的关注。回顾点云质量评估（PCQA）方法的发展历程，视觉质量通常通过利用单模态信息（即从2D投影或3D点云中提取）进行评估。2D投影包含丰富的纹理和语义信息，但高度依赖视点；而3D点云对几何畸变更敏感且具有视点不变性。因此，为充分利用点云与投影图像两种模态的优势，我们提出了一种基于多模态方式的新型无参考点云质量评估（NR-PCQA）指标。具体而言，我们将点云分割为子模型以表征局部几何畸变（如点偏移和下采样），随后将点云渲染为2D图像投影以提取纹理特征。为实现上述目标，子模型与投影图像分别通过基于点云和基于图像的神经网络进行编码，最后采用对称跨模态注意力机制融合多模态质量感知信息。实验结果表明，本方法在所有对比的最先进方法中表现最优，且显著领先于以往的NR-PCQA方法，充分验证了所提方法的有效性。代码已开源：https://github.com/zzc-1998/MM-PCQA。