Humans constantly contact objects to move and perform tasks. Thus, detecting human-object contact is important for building human-centered artificial intelligence. However, there exists no robust method to detect contact between the body and the scene from an image, and there exists no dataset to learn such a detector. We fill this gap with HOT ("Human-Object conTact"), a new dataset of human-object contacts for images. To build HOT, we use two data sources: (1) We use the PROX dataset of 3D human meshes moving in 3D scenes, and automatically annotate 2D image areas for contact via 3D mesh proximity and projection. (2) We use the V-COCO, HAKE and Watch-n-Patch datasets, and ask trained annotators to draw polygons for the 2D image areas where contact takes place. We also annotate the involved body part of the human body. We use our HOT dataset to train a new contact detector, which takes a single color image as input, and outputs 2D contact heatmaps as well as the body-part labels that are in contact. This is a new and challenging task that extends current foot-ground or hand-object contact detectors to the full generality of the whole body. The detector uses a part-attention branch to guide contact estimation through the context of the surrounding body parts and scene. We evaluate our detector extensively, and quantitative results show that our model outperforms baselines, and that all components contribute to better performance. Results on images from an online repository show reasonable detections and generalizability.

翻译：人类不断接触物体以进行移动和执行任务。因此，检测人-物接触对于构建以人为中心的人工智能具有重要意义。然而，目前尚无鲁棒方法可以从图像中检测身体与场景之间的接触，也不存在用于学习此类检测器的数据集。我们通过构建HOT（"人类-物体接触"）数据集来填补这一空白，该数据集专门针对图像中的人-物接触。为构建HOT，我们使用两个数据来源：（1）利用PROX数据集中的3D人体网格在3D场景中运动的数据，通过3D网格邻近性和投影自动标注2D图像区域的接触；（2）利用V-COCO、HAKE和Watch-n-Patch数据集，由经过训练的标注员绘制发生接触的2D图像区域多边形，并标注参与接触的人体部位。我们基于HOT数据集训练了一个新型接触检测器，该检测器以单一彩色图像为输入，输出2D接触热图以及处于接触状态的部位标签。这是一个全新的具有挑战性的任务，它将当前的脚-地面或手-物体接触检测扩展到全身接触的通用场景。该检测器采用部位注意力分支，通过周围身体部位和场景的上下文引导接触估计。我们对该检测器进行了全面评估，定量结果表明我们的模型优于基线方法，且所有组件均有助于提升性能。对在线仓库中图像的测试结果显示，该方法具有合理的检测效果和良好的泛化能力。