Point cloud is one of the most widely used digital representation formats for three-dimensional (3D) contents, the visual quality of which may suffer from noise and geometric shift distortions during the production procedure as well as compression and downsampling distortions during the transmission process. To tackle the challenge of point cloud quality assessment (PCQA), many PCQA methods have been proposed to evaluate the visual quality levels of point clouds by assessing the rendered static 2D projections. Although such projection-based PCQA methods achieve competitive performance with the assistance of mature image quality assessment (IQA) methods, they neglect that the 3D model is also perceived in a dynamic viewing manner, where the viewpoint is continually changed according to the feedback of the rendering device. Therefore, in this paper, we evaluate the point clouds from moving camera videos and explore the way of dealing with PCQA tasks via using video quality assessment (VQA) methods. First, we generate the captured videos by rotating the camera around the point clouds through several circular pathways. Then we extract both spatial and temporal quality-aware features from the selected key frames and the video clips through using trainable 2D-CNN and pre-trained 3D-CNN models respectively. Finally, the visual quality of point clouds is represented by the video quality values. The experimental results reveal that the proposed method is effective for predicting the visual quality levels of the point clouds and even competitive with full-reference (FR) PCQA methods. The ablation studies further verify the rationality of the proposed framework and confirm the contributions made by the quality-aware features extracted via the dynamic viewing manner. The code is available at https://github.com/zzc-1998/VQA_PC.
翻译:点云是三维内容最广泛使用的数字表示格式之一,其视觉质量在生产过程中可能因噪声和几何位移失真而受损,在传输过程中也可能因压缩和下采样失真而降低。为应对点云质量评估(PCQA)的挑战,已有多种PCQA方法通过评估渲染的静态二维投影来评价点云的视觉质量水平。尽管这类基于投影的PCQA方法借助成熟的图像质量评估(IQA)方法取得了有竞争力的性能,但它们忽略了三维模型通常以动态观看方式被感知——即视点会根据渲染设备的反馈持续变化。因此,本文从移动摄像机视频中评估点云,并探索利用视频质量评估(VQA)方法处理PCQA任务的新途径。首先,我们通过沿多条圆形路径绕点云旋转摄像机来生成捕获视频。然后,分别利用可训练的2D-CNN和预训练的3D-CNN模型从选定关键帧和视频片段中提取空间与时域质量感知特征。最终,点云的视觉质量由视频质量值表示。实验结果表明,所提方法能有效预测点云的视觉质量水平,甚至可与全参考(FR)PCQA方法相媲美。消融研究进一步验证了所提框架的合理性,并确认了通过动态观看方式提取的质量感知特征的贡献。代码开源于https://github.com/zzc-1998/VQA_PC。