We use our hands to interact with and to manipulate objects. Articulated objects are especially interesting since they often require the full dexterity of human hands to manipulate them. To understand, model, and synthesize such interactions, automatic and robust methods that reconstruct hands and articulated objects in 3D from a color image are needed. Existing methods for estimating 3D hand and object pose from images focus on rigid objects. In part, because such methods rely on training data and no dataset of articulated object manipulation exists. Consequently, we introduce ARCTIC - the first dataset of free-form interactions of hands and articulated objects. ARCTIC has 1.2M images paired with accurate 3D meshes for both hands and for objects that move and deform over time. The dataset also provides hand-object contact information. To show the value of our dataset, we perform two novel tasks on ARCTIC: (1) 3D reconstruction of two hands and an articulated object in interaction; (2) an estimation of dense hand-object relative distances, which we call interaction field estimation. For the first task, we present ArcticNet, a baseline method for the task of jointly reconstructing two hands and an articulated object from an RGB image. For interaction field estimation, we predict the relative distances from each hand vertex to the object surface, and vice versa. We introduce InterField, the first method that estimates such distances from a single RGB image. We provide qualitative and quantitative experiments for both tasks, and provide detailed analysis on the data. Code and data will be available at https://arctic.is.tue.mpg.de.

翻译：我们用手进行交互与操作物体。铰接物体尤为有趣，因其常需人类双手的完全灵巧性才能操控。为理解、建模并合成此类交互，需开发能从彩色图像中自动且稳健地重建双手及铰接物体三维结构的方法。现有从图像估计手与物体三维姿态的方法主要针对刚性物体，部分原因在于这些方法依赖训练数据，而目前尚无铰接物体操作的数据集。因此，我们提出ARCTIC——首个记录手部与铰接物体自由交互的数据集。ARCTIC包含120万张图像，每张均配有随时间运动变形的双手与物体的精确三维网格，并提供手-物接触信息。为展示数据集价值，我们在ARCTIC上开展两项新任务：(1) 交互中双手与铰接物体的三维重建；(2) 密集手-物相对距离估计，即交互场估计。针对第一项任务，我们提出ArcticNet作为从RGB图像联合重建双手与铰接物体的基线方法；针对交互场估计，我们预测每个手部顶点到物体表面的相对距离（反之亦然），并介绍InterField——首个从单张RGB图像估计此类距离的方法。我们为两项任务提供定性与定量实验，并对数据开展详细分析。代码及数据将发布于https://arctic.is.tue.mpg.de。