Hands are essential to human interaction, and exploring contact between hands and the world can promote comprehensive understanding of their function. Recently, there have been growing number of hand interaction datasets that cover interaction with object, other hand, scene, and body. Despite the significance of the task and increasing high-quality data, how to effectively learn dense hand contact estimation remains largely underexplored. There are two major challenges for learning dense hand contact estimation. First, there exists class imbalance issue from hand contact datasets where majority of regions are not in contact. Second, hand contact datasets contain spatial imbalance issue with most of hand contact exhibited in finger tips, resulting in challenges for generalization towards contacts in other hand regions. To tackle these issues, we present a framework that learns dense HAnd COntact estimation (HACO) from imbalanced data. To resolve the class imbalance issue, we introduce balanced contact sampling, which builds and samples from multiple sampling groups that fairly represent diverse contact statistics for both contact and non-contact vertices. Moreover, to address the spatial imbalance issue, we propose vertex-level class-balanced (VCB) loss, which incorporates spatially varying contact distribution by separately reweighting loss contribution of each vertex based on its contact frequency across dataset. As a result, we effectively learn to predict dense hand contact estimation with large-scale hand contact data without suffering from class and spatial imbalance issue. The codes are available at https://github.com/dqj5182/HACO_RELEASE.

翻译：手部在人类交互中至关重要，探索手部与外界之间的接触有助于全面理解其功能。近年来，涵盖手部与物体、另一只手、场景及身体交互的手部交互数据集日益增多。尽管该任务意义重大且高质量数据不断增长，如何有效学习密集手部接触估计仍存在大量未探索空间。学习密集手部接触估计面临两大挑战：首先，手部接触数据集中存在类别不平衡问题，即大部分区域未处于接触状态；其次，数据集存在空间不平衡问题，多数手部接触集中于指尖区域，导致模型难以泛化至手部其他区域的接触。为应对这些问题，我们提出一个从非平衡数据中学习密集手部接触估计（HACO）的框架。针对类别不平衡问题，我们提出平衡接触采样方法，通过构建并采样多个能公平代表接触与非接触顶点不同接触统计特征的采样组。此外，为解决空间不平衡问题，我们提出顶点级类别平衡（VCB）损失函数，该函数通过根据每个顶点在数据集中的接触频率单独重新加权其损失贡献，从而纳入空间变化的接触分布。最终，我们能够有效利用大规模手部接触数据预测密集手部接触估计，同时避免类别与空间不平衡问题的影响。代码发布于 https://github.com/dqj5182/HACO_RELEASE。