We present GS-ProCams, the first Gaussian Splatting-based framework for projector-camera systems (ProCams). GS-ProCams significantly enhances the efficiency of projection mapping (PM) that requires establishing geometric and radiometric mappings between the projector and the camera. Previous CNN-based ProCams are constrained to a specific viewpoint, limiting their applicability to novel perspectives. In contrast, NeRF-based ProCams support view-agnostic projection mapping, however, they require an additional colocated light source and demand significant computational and memory resources. To address this issue, we propose GS-ProCams that employs 2D Gaussian for scene representations, and enables efficient view-agnostic ProCams applications. In particular, we explicitly model the complex geometric and photometric mappings of ProCams using projector responses, the target surface's geometry and materials represented by Gaussians, and global illumination component. Then, we employ differentiable physically-based rendering to jointly estimate them from captured multi-view projections. Compared to state-of-the-art NeRF-based methods, our GS-ProCams eliminates the need for additional devices, achieving superior ProCams simulation quality. It is also 600 times faster and uses only 1/10 of the GPU memory.

翻译：我们提出了GS-ProCams，这是首个基于高斯溅射的投影仪-相机系统框架。GS-ProCams显著提升了投影映射的效率，该过程需要在投影仪与相机之间建立几何与辐射度映射。以往基于CNN的投影仪-相机系统受限于特定视角，难以应用于新视角场景。相比之下，基于NeRF的投影仪-相机系统虽支持视角无关的投影映射，但需要额外的共置光源，且对计算与内存资源需求较高。为解决这一问题，我们提出GS-ProCams，其采用二维高斯函数进行场景表示，实现了高效的视角无关投影仪-相机系统应用。具体而言，我们通过投影仪响应、高斯函数表示的目标表面几何与材质属性以及全局光照分量，显式建模了投影仪-相机系统中复杂的几何与光度映射关系。随后，我们采用基于物理的可微分渲染方法，从捕获的多视角投影数据中联合估计这些参数。与当前最先进的基于NeRF的方法相比，我们的GS-ProCams无需额外设备即可实现更优的投影仪-相机系统仿真质量，其运行速度提升600倍，且仅需1/10的GPU显存。