We consider the problem of physically-based inverse rendering using 3D Gaussian Splatting (3DGS) representations. While recent 3DGS methods have achieved remarkable results in novel view synthesis (NVS), accurately capturing high-fidelity geometry, physically interpretable materials and lighting remains challenging, as it requires precise geometry modeling to provide accurate surface normals, along with physically-based rendering (PBR) techniques to ensure correct material and lighting disentanglement. Previous 3DGS methods resort to approximating surface normals, but often struggle with noisy local geometry, leading to inaccurate normal estimation and suboptimal material-lighting decomposition. In this paper, we introduce GeoSplatting, a novel hybrid representation that augments 3DGS with explicit geometric guidance and differentiable PBR equations. Specifically, we bridge isosurface and 3DGS together, where we first extract isosurface mesh from a scalar field, then convert it into 3DGS points and formulate PBR equations for them in a fully differentiable manner. In GeoSplatting, 3DGS is grounded on the mesh geometry, enabling precise surface normal modeling, which facilitates the use of PBR frameworks for material decomposition. This approach further maintains the efficiency and quality of NVS from 3DGS while ensuring accurate geometry from the isosurface. Comprehensive evaluations across diverse datasets demonstrate the superiority of GeoSplatting, consistently outperforming existing methods both quantitatively and qualitatively.

翻译：本文研究了基于物理的逆向渲染问题，并采用3D高斯泼溅（3DGS）表示方法。尽管近期3DGS方法在新视角合成（NVS）方面取得了显著成果，但精确捕捉高保真几何结构、物理可解释材质与光照仍具挑战性，因为这需要精确的几何建模以提供准确的表面法线，同时需结合基于物理的渲染（PBR）技术以确保正确的材质与光照解耦。现有3DGS方法多采用近似表面法线方案，但常受噪声局部几何影响，导致法线估计不准确及材质-光照分解结果欠佳。本文提出GeoSplatting——一种融合显式几何引导与可微分PBR方程的新型混合表示方法。具体而言，我们将等值面与3DGS相融合：首先从标量场提取等值面网格，将其转换为3DGS点集，并以完全可微分形式构建PBR方程。在GeoSplatting中，3DGS以网格几何为基础，实现了精确的表面法线建模，从而为材质分解提供了PBR框架支持。该方法在保持3DGS高效NVS能力与渲染质量的同时，确保了等值面几何的精确性。跨多数据集的综合评估表明，GeoSplatting在定量与定性评估中均持续优于现有方法。