We present Mono-STAR, the first real-time 3D reconstruction system that simultaneously supports semantic fusion, fast motion tracking, non-rigid object deformation, and topological change under a unified framework. The proposed system solves a new optimization problem incorporating optical-flow-based 2D constraints to deal with fast motion and a novel semantic-aware deformation graph (SAD-graph) for handling topology change. We test the proposed system under various challenging scenes and demonstrate that it significantly outperforms existing state-of-the-art methods.

翻译：我们提出了Mono-STAR，这是首个在统一框架下同时支持语义融合、快速运动跟踪、非刚性物体变形与拓扑变化的实时三维重建系统。该系统通过求解一个融合了基于光流的二维约束以应对快速运动的新型优化问题，并引入语义感知变形图（SAD-graph）来处理拓扑变化。我们在多种具有挑战性的场景下测试了所提系统，结果表明其显著优于现有最先进方法。