While novel view synthesis (NVS) has made substantial progress in 3D computer vision, it typically requires an initial estimation of camera intrinsics and extrinsics from dense viewpoints. This pre-processing is usually conducted via a Structure-from-Motion (SfM) pipeline, a procedure that can be slow and unreliable, particularly in sparse-view scenarios with insufficient matched features for accurate reconstruction. In this work, we integrate the strengths of point-based representations (e.g., 3D Gaussian Splatting, 3D-GS) with end-to-end dense stereo models (DUSt3R) to tackle the complex yet unresolved issues in NVS under unconstrained settings, which encompasses pose-free and sparse view challenges. Our framework, InstantSplat, unifies dense stereo priors with 3D-GS to build 3D Gaussians of large-scale scenes from sparseview & pose-free images in less than 1 minute. Specifically, InstantSplat comprises a Coarse Geometric Initialization (CGI) module that swiftly establishes a preliminary scene structure and camera parameters across all training views, utilizing globally-aligned 3D point maps derived from a pre-trained dense stereo pipeline. This is followed by the Fast 3D-Gaussian Optimization (F-3DGO) module, which jointly optimizes the 3D Gaussian attributes and the initialized poses with pose regularization. Experiments conducted on the large-scale outdoor Tanks & Temples datasets demonstrate that InstantSplat significantly improves SSIM (by 32%) while concurrently reducing Absolute Trajectory Error (ATE) by 80%. These establish InstantSplat as a viable solution for scenarios involving posefree and sparse-view conditions. Project page: instantsplat.github.io.

翻译：尽管新视角合成在3D计算机视觉领域取得了显著进展，但其通常需要从密集视角预先估计相机内参与外参。这一预处理步骤通常通过运动恢复结构（SfM）流水线完成，在稀疏视角场景中，由于匹配特征不足，该过程可能既缓慢又不可靠。本研究通过融合基于点表示（如3D高斯泼溅，3D-GS）与端到端密集立体模型（DUSt3R）的优势，解决非约束设置下（包括无位姿和稀疏视角挑战）新视角合成中复杂且尚未解决的问题。我们的框架InstantSplat统一了密集立体先验与3D-GS，可在1分钟内从稀疏无位姿图像构建大规模场景的3D高斯模型。具体而言，InstantSplat包含粗几何初始化（CGI）模块，该模块利用预训练密集立体流水线生成的全局对齐3D点图，快速建立所有训练视图的初始场景结构与相机参数。随后通过快速3D高斯优化（F-3DGO）模块，联合优化3D高斯属性与初始化位姿，并引入位姿正则化。在大型室外Tanks & Temples数据集上的实验表明，InstantSplat将SSIM显著提升32%，同时将绝对轨迹误差（ATE）降低80%。这确立了InstantSplat作为无位姿和稀疏视角场景下可行方案的定位。项目页面：instantsplat.github.io。