Prehensile autonomous manipulation, such as peg insertion, tool use, or assembly, require precise in-hand understanding of the object pose and the extrinsic contacts made during interactions. Providing accurate estimation of pose and contacts is challenging. Tactile sensors can provide local geometry at the sensor and force information about the grasp, but the locality of sensing means resolving poses and contacts from tactile alone is often an ill-posed problem, as multiple configurations can be consistent with the observations. Adding visual feedback can help resolve ambiguities, but can suffer from noise and occlusions. In this work, we propose a method that pairs local observations from sensing with the physical constraints of contact. We propose a set of factors that ensure local consistency with tactile observations as well as enforcing physical plausibility, namely, that the estimated pose and contacts must respect the kinematic and force constraints of quasi-static rigid body interactions. We formalize our problem as a factor graph, allowing for efficient estimation. In our experiments, we demonstrate that our method outperforms existing geometric and contact-informed estimation pipelines, especially when only tactile information is available. Video results can be found at https://tacgraph.github.io/.

翻译：灵巧自主操作（如销钉插入、工具使用或装配）需要精确掌握手内物体的姿态以及交互过程中产生的外部接触。提供准确的姿态与接触估计具有挑战性。触觉传感器能够提供传感器局部的几何信息及抓握力信息，但感知的局部性意味着仅凭触觉数据解析姿态与接触常构成不适定问题，因为多种构型都可能与观测结果一致。引入视觉反馈虽有助于消除歧义，但易受噪声与遮挡干扰。本研究提出一种将局部感知观测与接触物理约束相结合的方法。我们设计了一组因子，既保证与触觉观测的局部一致性，又强化物理合理性——即估计的姿态与接触必须满足准静态刚体相互作用的运动学与力学约束。我们将该问题形式化为因子图，从而实现高效估计。实验表明，本方法在现有几何与接触感知估计流程中表现优异，尤其在仅依赖触觉信息时优势显著。视频结果详见 https://tacgraph.github.io/。