Holistic 3D human-scene reconstruction is a crucial and emerging research area in robot perception. A key challenge in holistic 3D human-scene reconstruction is to generate a physically plausible 3D scene from a single monocular RGB image. The existing research mainly proposes optimization-based approaches for reconstructing the scene from a sequence of RGB frames with explicitly defined physical laws and constraints between different scene elements (humans and objects). However, it is hard to explicitly define and model every physical law in every scenario. This paper proposes using an implicit feature representation of the scene elements to distinguish a physically plausible alignment of humans and objects from an implausible one. We propose using a graph-based holistic representation with an encoded physical representation of the scene to analyze the human-object and object-object interactions within the scene. Using this graphical representation, we adversarially train our model to learn the feasible alignments of the scene elements from the training data itself without explicitly defining the laws and constraints between them. Unlike the existing inference-time optimization-based approaches, we use this adversarially trained model to produce a per-frame 3D reconstruction of the scene that abides by the physical laws and constraints. Our learning-based method achieves comparable 3D reconstruction quality to existing optimization-based holistic human-scene reconstruction methods and does not need inference time optimization. This makes it better suited when compared to existing methods, for potential use in robotic applications, such as robot navigation, etc.

翻译：整体三维人-场景重建是机器人感知领域中一项关键且新兴的研究方向。该领域的核心挑战在于如何从单张单目RGB图像生成物理合理的三维场景。现有研究主要采用基于优化的方法，通过显式定义物理定律及不同场景元素（人与物体）之间的约束，从RGB图像序列中重建场景。然而，显式定义并建模所有场景下的物理定律极为困难。本文提出利用场景元素的隐式特征表示，区分人与物体的物理合理对齐与不合理对齐。我们采用基于图的整体表示方法，结合场景编码的物理特征表示，分析场景中的人-物交互及物-物交互。通过这种图表示，我们以对抗方式训练模型，使其从训练数据本身学习场景元素的可行对齐方式，而无需显式定义元素间的物理定律与约束。与现有基于推理时优化的方法不同，我们利用该对抗训练模型逐帧生成符合物理定律与约束的三维场景重建结果。本基于学习的方法在三维重建质量上可与现有基于优化的整体人-场景重建方法相媲美，且无需推理时优化。相较于现有方法，该方法更适用于机器人导航等机器人应用场景。