We present LRM-Zero, a Large Reconstruction Model (LRM) trained entirely on synthesized 3D data, achieving high-quality sparse-view 3D reconstruction. The core of LRM-Zero is our procedural 3D dataset, Zeroverse, which is automatically synthesized from simple primitive shapes with random texturing and augmentations (e.g., height fields, boolean differences, and wireframes). Unlike previous 3D datasets (e.g., Objaverse) which are often captured or crafted by humans to approximate real 3D data, Zeroverse completely ignores realistic global semantics but is rich in complex geometric and texture details that are locally similar to or even more intricate than real objects. We demonstrate that our LRM-Zero, trained with our fully synthesized Zeroverse, can achieve high visual quality in the reconstruction of real-world objects, competitive with models trained on Objaverse. We also analyze several critical design choices of Zeroverse that contribute to LRM-Zero's capability and training stability. Our work demonstrates that 3D reconstruction, one of the core tasks in 3D vision, can potentially be addressed without the semantics of real-world objects. The Zeroverse's procedural synthesis code and interactive visualization are available at: https://desaixie.github.io/lrm-zero/.
翻译:我们提出LRM-Zero,一种完全基于合成3D数据训练的大规模重建模型(LRM),实现了高质量稀疏视角3D重建。LRM-Zero的核心是我们程序化生成的3D数据集Zeroverse,该数据集通过随机纹理与增强技术(如高度场、布尔差分和线框结构)从简单基本几何形状自动合成。与以往常由人工采集或制作以逼近真实3D数据的数据集(如Objaverse)不同,Zeroverse完全忽略全局真实语义,但富含与真实物体局部相似甚至更为复杂的几何与纹理细节。实验表明,基于完全合成的Zeroverse训练的LRM-Zero,在真实世界物体重建中可获得与基于Objaverse训练的模型相媲美的高视觉质量。我们进一步分析了Zeroverse中若干关键设计选择对LRM-Zero性能及训练稳定性的贡献。本工作证明,作为3D视觉核心任务之一的3D重建,有望在不依赖真实世界物体语义的条件下实现。Zeroverse的程序化合成代码与交互式可视化已开源:https://desaixie.github.io/lrm-zero/。