Currently, 3D rendering for large-scale free camera trajectories, namely, arbitrary input camera trajectories, poses significant challenges: 1) The distribution and observation angles of the cameras are irregular, and various types of scenes are included in the free trajectories; 2) Processing the entire point cloud and all images at once for large-scale scenes requires a substantial amount of GPU memory. This paper presents a Toy-GS method for accurately rendering large-scale free camera trajectories. Specifically, we propose an adaptive spatial division approach for free trajectories to divide cameras and the sparse point cloud of the entire scene into various regions according to camera poses. Training each local Gaussian in parallel for each area enables us to concentrate on texture details and minimize GPU memory usage. Next, we use the multi-view constraint and position-aware point adaptive control (PPAC) to improve the rendering quality of texture details. In addition, our regional fusion approach combines local and global Gaussians to enhance rendering quality with an increasing number of divided areas. Extensive experiments have been carried out to confirm the effectiveness and efficiency of Toy-GS, leading to state-of-the-art results on two public large-scale datasets as well as our SCUTic dataset. Our proposal demonstrates an enhancement of 1.19 dB in PSNR and conserves 7 G of GPU memory when compared to various benchmarks.
翻译:当前,针对大规模自由相机轨迹(即任意输入相机轨迹)的3D渲染面临重大挑战:1)相机分布与观测角度不规则,且自由轨迹中包含多种场景类型;2)对于大规模场景,一次性处理全部点云与所有图像需要消耗大量GPU显存。本文提出一种用于精确渲染大规模自由相机轨迹的Toy-GS方法。具体而言,我们针对自由轨迹提出一种自适应空间划分方法,依据相机位姿将相机与整个场景的稀疏点云划分至不同区域。通过并行训练每个区域的局部高斯分布,我们得以聚焦纹理细节并最小化GPU显存占用。随后,我们利用多视图约束与位置感知点自适应控制(PPAC)来提升纹理细节的渲染质量。此外,我们的区域融合方法通过结合局部与全局高斯分布,在划分区域数量增加时进一步提升渲染质量。大量实验证实了Toy-GS的有效性与高效性,在两个公开大规模数据集及我们的SCUTic数据集上均取得了最先进的结果。与多种基准方法相比,本方法在PSNR指标上提升了1.19 dB,并节省了7 GB的GPU显存。