Many autonomous robotic applications require object-level understanding when deployed. Actively reconstructing objects of interest, i.e. objects with specific semantic meanings, is therefore relevant for a robot to perform downstream tasks in an initially unknown environment. In this work, we propose a novel framework for semantic-targeted active reconstruction using posed RGB-D measurements and 2D semantic labels as input. The key components of our framework are a semantic implicit neural representation and a compatible planning utility function based on semantic rendering and uncertainty estimation, enabling adaptive view planning to target objects of interest. Our planning approach achieves better reconstruction performance in terms of mesh and novel view rendering quality compared to implicit reconstruction baselines that do not consider semantics for view planning. Our framework further outperforms a state-of-the-art semantic-targeted active reconstruction pipeline based on explicit maps, justifying our choice of utilising implicit neural representations to tackle semantic-targeted active reconstruction problems.

翻译：许多自主机器人应用在部署时需要理解物体层级的信息。因此，主动重建感兴趣的物体（即具有特定语义含义的物体）对于机器人在初始未知环境中执行下游任务至关重要。本文提出了一种新颖的语义目标驱动的主动重建框架，以带有位姿的RGB-D测量值和二维语义标签作为输入。该框架的核心组件包括语义隐式神经表示和基于语义渲染与不确定性估计的兼容规划效用函数，从而能够自适应地规划视角以聚焦于感兴趣的物体。与不考虑语义进行视角规划的隐式重建基线方法相比，我们的规划方法在网格和新视角渲染质量方面取得了更优的重建性能。此外，我们的框架进一步超越了基于显式地图的最先进语义目标驱动主动重建流程，验证了利用隐式神经表示解决语义目标驱动主动重建问题的优势。