We present Neural Contact Fields, a method that brings together neural fields and tactile sensing to address the problem of tracking extrinsic contact between object and environment. Knowing where the external contact occurs is a first step towards methods that can actively control it in facilitating downstream manipulation tasks. Prior work for localizing environmental contacts typically assume a contact type (e.g. point or line), does not capture contact/no-contact transitions, and only works with basic geometric-shaped objects. Neural Contact Fields are the first method that can track arbitrary multi-modal extrinsic contacts without making any assumptions about the contact type. Our key insight is to estimate the probability of contact for any 3D point in the latent space of object shapes, given vision-based tactile inputs that sense the local motion resulting from the external contact. In experiments, we find that Neural Contact Fields are able to localize multiple contact patches without making any assumptions about the geometry of the contact, and capture contact/no-contact transitions for known categories of objects with unseen shapes in unseen environment configurations. In addition to Neural Contact Fields, we also release our YCB-Extrinsic-Contact dataset of simulated extrinsic contact interactions to enable further research in this area. Project page: https://github.com/carolinahiguera/NCF

翻译：我们提出神经接触场（Neural Contact Fields）方法，该方法融合神经场与触觉感知，以解决物体与环境间外部接触的跟踪问题。确定外部接触发生的位置，是开发能主动控制接触以促进下游操作任务的方法的首要步骤。现有环境接触定位工作通常假设接触类型（如点接触或线接触），无法捕捉接触/非接触转换，且仅适用于基本几何形状的物体。神经接触场是首个无需对接触类型做任何假设即可跟踪任意多模态外部接触的方法。我们的核心洞察在于：基于视觉触觉输入（感知由外部接触导致的局部运动），在物体形状的潜在空间中估计任意3D点发生接触的概率。实验表明，神经接触场能够在不对接触几何形状做任何假设的前提下定位多个接触区域，并在未见过的环境配置中，对已知类别但形状未知的物体捕捉接触/非接触转换。除神经接触场外，我们还发布了模拟外部接触交互的YCB-Extrinsic-Contact数据集，以推动该领域的进一步研究。项目页面：https://github.com/carolinahiguera/NCF