Generating high-quality 3D content requires models capable of learning robust distributions of complex scenes and the real-world objects within them. Recent Gaussian-based 3D reconstruction techniques have achieved impressive results in recovering high-fidelity 3D assets from sparse input images by predicting 3D Gaussians in a feed-forward manner. However, these techniques often lack the extensive priors and expressiveness offered by Diffusion Models. On the other hand, 2D Diffusion Models, which have been successfully applied to denoise multiview images, show potential for generating a wide range of photorealistic 3D outputs but still fall short on explicit 3D priors and consistency. In this work, we aim to bridge these two approaches by introducing DSplats, a novel method that directly denoises multiview images using Gaussian Splat-based Reconstructors to produce a diverse array of realistic 3D assets. To harness the extensive priors of 2D Diffusion Models, we incorporate a pretrained Latent Diffusion Model into the reconstructor backbone to predict a set of 3D Gaussians. Additionally, the explicit 3D representation embedded in the denoising network provides a strong inductive bias, ensuring geometrically consistent novel view generation. Our qualitative and quantitative experiments demonstrate that DSplats not only produces high-quality, spatially consistent outputs, but also sets a new standard in single-image to 3D reconstruction. When evaluated on the Google Scanned Objects dataset, DSplats achieves a PSNR of 20.38, an SSIM of 0.842, and an LPIPS of 0.109.

翻译：生成高质量的三维内容需要模型能够学习复杂场景及其内部真实物体的稳健分布。近期基于高斯的三维重建技术通过前馈方式预测三维高斯，在从稀疏输入图像恢复高保真三维资产方面取得了令人瞩目的成果。然而，这些技术通常缺乏扩散模型所提供的广泛先验和表达能力。另一方面，已成功应用于多视角图像去噪的二维扩散模型，在生成多样化、照片级真实感三维输出方面展现出潜力，但在显式三维先验和一致性方面仍存在不足。本研究旨在通过引入DSplats来桥接这两种方法：这是一种新颖的方法，它使用基于高斯泼溅的重建器直接对多视角图像进行去噪，从而生成多样化的逼真三维资产。为了利用二维扩散模型的广泛先验，我们将一个预训练的潜在扩散模型整合到重建器主干中，以预测一组三维高斯。此外，嵌入去噪网络中的显式三维表示提供了强大的归纳偏置，确保了几何一致的新视角生成。我们的定性和定量实验表明，DSplats不仅能产生高质量、空间一致的输出，还为单图像到三维重建设立了新标准。在Google Scanned Objects数据集上的评估显示，DSplats取得了20.38的PSNR、0.842的SSIM和0.109的LPIPS。