The dominant majority of 3D models that appear in gaming, VR/AR, and those we use to train geometric deep learning algorithms are incomplete, since they are modeled as surface meshes and missing their interior structures. We present a learning framework to recover the shape interiors (RoSI) of existing 3D models with only their exteriors from multi-view and multi-articulation images. Given a set of RGB images that capture a target 3D object in different articulated poses, possibly from only few views, our method infers the interior planes that are observable in the input images. Our neural architecture is trained in a category-agnostic manner and it consists of a motion-aware multi-view analysis phase including pose, depth, and motion estimations, followed by interior plane detection in images and 3D space, and finally multi-view plane fusion. In addition, our method also predicts part articulations and is able to realize and even extrapolate the captured motions on the target 3D object. We evaluate our method by quantitative and qualitative comparisons to baselines and alternative solutions, as well as testing on untrained object categories and real image inputs to assess its generalization capabilities.

翻译：绝大多数出现在游戏、VR/AR中的三维模型，以及我们用于训练几何深度学习算法的模型都是不完整的，因为它们被建模为表面网格且缺少内部结构。我们提出一个学习框架，用于从多视角和多关节图像中恢复现有三维模型的形状内部结构（RoSI），这些模型仅具有外部表面。给定一组在不同关节点姿态下捕获目标三维物体的RGB图像（可能仅来自少数视角），我们的方法能推断输入图像中可观测的内部平面。我们的神经架构以类别无关的方式训练，包含运动感知的多视角分析阶段（包括姿态、深度和运动估计），随后在图像和三维空间中进行内部平面检测，最后进行多视角平面融合。此外，我们的方法还能预测部件关节点，并能够实现甚至外推目标三维物体上捕获的运动。通过定量和定性与基准方法及替代方案对比，以及在未训练物体类别和真实图像输入上的测试，我们评估了该方法的表现能力。