We present GS-ProCams, the first Gaussian Splatting-based framework for projector-camera systems (ProCams). GS-ProCams is not only view-agnostic but also significantly enhances the efficiency of projection mapping (PM) that requires establishing geometric and radiometric mappings between the projector and the camera. Previous CNN-based ProCams are constrained to a specific viewpoint, limiting their applicability to novel perspectives. In contrast, NeRF-based ProCams support view-agnostic projection mapping, however, they require an additional co-located light source and demand significant computational and memory resources. To address this issue, we propose GS-ProCams that employs 2D Gaussian for scene representations, and enables efficient view-agnostic ProCams applications. In particular, we explicitly model the complex geometric and photometric mappings of ProCams using projector responses, the projection surface's geometry and materials represented by Gaussians, and the global illumination component. Then, we employ differentiable physically-based rendering to jointly estimate them from captured multi-view projections. Compared to state-of-the-art NeRF-based methods, our GS-ProCams eliminates the need for additional devices, achieving superior ProCams simulation quality. It also uses only 1/10 of the GPU memory for training and is 900 times faster in inference speed. Please refer to our project page for the code and dataset: https://realqingyue.github.io/GS-ProCams/.

翻译：我们提出了GS-ProCams，这是首个基于高斯泼溅的投影仪-相机系统框架。GS-ProCams不仅与视角无关，还显著提升了投影映射的效率，该过程需要在投影仪与相机之间建立几何与辐射度映射。以往基于CNN的ProCams受限于特定视点，难以应用于新视角。相比之下，基于NeRF的ProCams虽支持视角无关的投影映射，但需要额外的共置光源，且对计算与内存资源需求较高。为解决此问题，我们提出GS-ProCams，它采用二维高斯进行场景表示，实现了高效且视角无关的ProCams应用。具体而言，我们通过投影仪响应、由高斯表示的投影表面几何与材质属性以及全局光照分量，显式建模了ProCams中复杂的几何与光度映射关系。随后，我们采用基于物理的可微分渲染方法，从捕获的多视角投影图像中联合估计这些参数。与当前最先进的基于NeRF的方法相比，我们的GS-ProCams无需额外设备即可实现更优的ProCams仿真质量，训练时仅需1/10的GPU内存，且推理速度提升900倍。代码与数据集详见项目页面：https://realqingyue.github.io/GS-ProCams/。