3D Gaussian Splatting (3DGS) has recently emerged as a state-of-the-art 3D reconstruction and rendering technique due to its high-quality results and fast training and rendering time. However, pixels covered by the same Gaussian are always shaded in the same color up to a Gaussian falloff scaling factor. Furthermore, the finest geometric detail any individual Gaussian can represent is a simple ellipsoid. These properties of 3DGS greatly limit the expressivity of individual Gaussian primitives. To address these issues, we draw inspiration from texture and alpha mapping in traditional graphics and integrate it with 3DGS. Specifically, we propose a new generalized Gaussian appearance representation that augments each Gaussian with alpha~(A), RGB, or RGBA texture maps to model spatially varying color and opacity across the extent of each Gaussian. As such, each Gaussian can represent a richer set of texture patterns and geometric structures, instead of just a single color and ellipsoid as in naive Gaussian Splatting. Surprisingly, we found that the expressivity of Gaussians can be greatly improved by using alpha-only texture maps, and further augmenting Gaussians with RGB texture maps achieves the highest expressivity. We validate our method on a wide variety of standard benchmark datasets and our own custom captures at both the object and scene levels. We demonstrate image quality improvements over existing methods while using a similar or lower number of Gaussians.

翻译：三维高斯泼溅（3DGS）凭借其高质量结果以及快速的训练与渲染速度，近期已成为最先进的三维重建与渲染技术。然而，被同一高斯覆盖的像素始终呈现相同颜色（仅受高斯衰减缩放因子影响）。此外，任何单个高斯所能表示的最精细几何细节仅为简单椭球体。3DGS的这些特性极大限制了单个高斯基元的表达能力。为解决这些问题，我们从传统图形学中的纹理与透明度映射汲取灵感，并将其与3DGS相结合。具体而言，我们提出一种新的广义高斯外观表示方法，通过为每个高斯附加透明度（A）、RGB或RGBA纹理贴图来建模每个高斯空间范围内的颜色与不透明度变化。由此，每个高斯可以表示更丰富的纹理模式与几何结构，而非如原始高斯泼溅中仅能表示单一颜色与椭球体。令人惊讶的是，我们发现仅使用透明度纹理贴图即可显著提升高斯的表达能力，而进一步为高斯附加RGB纹理贴图则可达到最高表达能力。我们在多种标准基准数据集及自主采集的对象级与场景级数据上验证了本方法，证明在保持相近或更少高斯数量的同时，图像质量优于现有方法。