Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. Our tactile sensing provides us with a lot of information regarding contact formations as well as geometric information about objects during any interaction. With this motivation, vision-based tactile sensors are being widely used for various robotic perception and control tasks. In this paper, we present a method for interactive perception using vision-based tactile sensors for a part mating task, where a robot can use tactile sensors and a feedback mechanism using a particle filter to incrementally improve its estimate of objects (pegs and holes) that fit together. To do this, we first train a deep neural network that makes use of tactile images to predict the probabilistic correspondence between arbitrarily shaped objects that fit together. The trained model is used to design a particle filter which is used twofold. First, given one partial (or non-unique) observation of the hole, it incrementally improves the estimate of the correct peg by sampling more tactile observations. Second, it selects the next action for the robot to sample the next touch (and thus image) which results in maximum uncertainty reduction to minimize the number of interactions during the perception task. We evaluate our method on several part-mating tasks with novel objects using a robot equipped with a vision-based tactile sensor. We also show the efficiency of the proposed action selection method against a naive method. See supplementary video at https://www.youtube.com/watch?v=jMVBg_e3gLw .

翻译：人类依赖触觉与触觉感知完成大量灵巧操作任务。在交互过程中，触觉感知不仅提供接触构型信息，还传递物体的几何特征。受此启发，基于视觉的触觉传感器正被广泛应用于机器人感知与控制任务。本文提出一种基于视觉触觉传感器的交互感知方法，用于零件装配任务——机器人可通过触觉传感器与基于粒子滤波的反馈机制，逐步优化对可配合物体（销与孔）的位姿估计。为此，我们首先训练一个深度神经网络，利用触觉图像预测任意形状可配合物体间的概率对应关系。该训练模型被用于设计粒子滤波器，其功能体现在两方面：第一，在给定单个不完整（或非唯一）的孔观测时，通过采集更多触觉观测逐步优化正确销的估计；第二，为机器人选择下一动作以采样下一个触觉（及对应图像），从而最大化不确定性降低，最小化感知任务中的交互次数。我们在一台配备视觉触觉传感器的机器人上，针对多种新型物体的零件装配任务评估了该方法，并展示了所提动作选择方法相较于朴素方法的效率优势。补充视频见 https://www.youtube.com/watch?v=jMVBg_e3gLw 。