We introduce an active 3D reconstruction method which integrates visual perception, robot-object interaction, and 3D scanning to recover both the exterior and interior, i.e., unexposed, geometries of a target 3D object. Unlike other works in active vision which focus on optimizing camera viewpoints to better investigate the environment, the primary feature of our reconstruction is an analysis of the interactability of various parts of the target object and the ensuing part manipulation by a robot to enable scanning of occluded regions. As a result, an understanding of part articulations of the target object is obtained on top of complete geometry acquisition. Our method operates fully automatically by a Fetch robot with built-in RGBD sensors. It iterates between interaction analysis and interaction-driven reconstruction, scanning and reconstructing detected moveable parts one at a time, where both the articulated part detection and mesh reconstruction are carried out by neural networks. In the final step, all the remaining, non-articulated parts, including all the interior structures that had been exposed by prior part manipulations and subsequently scanned, are reconstructed to complete the acquisition. We demonstrate the performance of our method via qualitative and quantitative evaluation, ablation studies, comparisons to alternatives, as well as experiments in a real environment.

翻译：我们提出一种主动三维重建方法，该方法整合视觉感知、机器人-物体交互与三维扫描技术，旨在恢复目标三维物体的外部几何与内部（即未暴露）几何结构。与现有专注于优化相机视角以更好探查环境的主动视觉研究不同，本方法的核心特征在于：通过分析目标物体各部件可操控性，进而由机器人执行部件操纵以扫描遮挡区域。最终在完整几何采集的基础上，实现对目标物体部件运动机理的认知。本方法基于配备RGBD传感器的Fetch机器人全自动运行，通过交互分析与交互驱动重建的迭代过程，逐一对检测到的可运动部件进行扫描与重建，其中铰接部件检测与网格重建均由神经网络实现。最后阶段，对所有剩余非铰接部件（包括先前部件操纵暴露并完成扫描的内部结构）进行重建以完成采集。我们通过定性定量评估、消融实验、方法对比及真实环境实验验证了本方法的有效性。