We present GSD, a diffusion model approach based on Gaussian Splatting (GS) representation for 3D object reconstruction from a single view. Prior works suffer from inconsistent 3D geometry or mediocre rendering quality due to improper representations. We take a step towards resolving these shortcomings by utilizing the recent state-of-the-art 3D explicit representation, Gaussian Splatting, and an unconditional diffusion model. This model learns to generate 3D objects represented by sets of GS ellipsoids. With these strong generative 3D priors, though learning unconditionally, the diffusion model is ready for view-guided reconstruction without further model fine-tuning. This is achieved by propagating fine-grained 2D features through the efficient yet flexible splatting function and the guided denoising sampling process. In addition, a 2D diffusion model is further employed to enhance rendering fidelity, and improve reconstructed GS quality by polishing and re-using the rendered images. The final reconstructed objects explicitly come with high-quality 3D structure and texture, and can be efficiently rendered in arbitrary views. Experiments on the challenging real-world CO3D dataset demonstrate the superiority of our approach. Project page: $\href{https://yxmu.foo/GSD/}{\text{this https URL}}$

翻译：本文提出GSD，一种基于高斯溅射表示的单视图三维物体重建扩散模型方法。现有方法因表示方式不当常存在三维几何不一致或渲染质量平庸的问题。我们通过采用当前最先进的三维显式表示——高斯溅射与无条件扩散模型，朝着解决这些缺陷迈进一步。该模型学习生成由GS椭球集合表示的三维物体。凭借这些强大的生成式三维先验，尽管采用无条件学习，该扩散模型无需额外微调即可实现视图引导重建。这是通过高效灵活的高斯溅射函数与引导去噪采样过程传播细粒度二维特征实现的。此外，本文进一步采用二维扩散模型增强渲染保真度，并通过优化与复用渲染图像提升重建GS质量。最终重建物体显式具备高质量三维结构与纹理，并可在任意视角高效渲染。在具有挑战性的真实世界CO3D数据集上的实验证明了本方法的优越性。项目页面：$\href{https://yxmu.foo/GSD/}{\text{此 https 链接}}$