We propose GS-LRM, a scalable large reconstruction model that can predict high-quality 3D Gaussian primitives from 2-4 posed sparse images in 0.23 seconds on single A100 GPU. Our model features a very simple transformer-based architecture; we patchify input posed images, pass the concatenated multi-view image tokens through a sequence of transformer blocks, and decode final per-pixel Gaussian parameters directly from these tokens for differentiable rendering. In contrast to previous LRMs that can only reconstruct objects, by predicting per-pixel Gaussians, GS-LRM naturally handles scenes with large variations in scale and complexity. We show that our model can work on both object and scene captures by training it on Objaverse and RealEstate10K respectively. In both scenarios, the models outperform state-of-the-art baselines by a wide margin. We also demonstrate applications of our model in downstream 3D generation tasks. Our project webpage is available at: https://sai-bi.github.io/project/gs-lrm/ .

翻译：我们提出GS-LRM，一种可扩展的大规模重建模型，能在单块A100 GPU上仅用0.23秒从2-4张稀疏带位姿图像预测出高质量的三维高斯基元。该模型采用极简的基于Transformer架构：将输入带位姿图像分块化处理，通过串联多视角图像令牌并经过一系列Transformer块传递，最终直接从这些令牌解码出逐像素高斯参数以支持可微渲染。与先前仅能重建物体的LRMs不同，通过预测逐像素高斯参数，GS-LRM能够自然处理具有尺度和复杂度巨大差异的场景。我们通过在Objaverse和RealEstate10K数据集上分别训练模型，证明了其可同时适用于物体与场景捕获任务。在这两种场景下，该模型均以显著优势超越现有最优基线方法。我们还展示了模型在下游三维生成任务中的应用。项目网页详见：https://sai-bi.github.io/project/gs-lrm/。