Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. Our tactile sensing provides us with a lot of information regarding contact formations as well as geometric information about objects during any interaction. With this motivation, vision-based tactile sensors are being widely used for various robotic perception and control tasks. In this paper, we present a method for interactive perception using vision-based tactile sensors for multi-object assembly. In particular, we are interested in tactile perception during part mating, where a robot can use tactile sensors and a feedback mechanism using particle filter to incrementally improve its estimate of objects that fit together for assembly. To do this, we first train a deep neural network that makes use of tactile images to predict the probabilistic correspondence between arbitrarily shaped objects that fit together. The trained model is used to design a particle filter which is used twofold. First, given one partial (or non-unique) observation of the hole, it incrementally improves the estimate of the correct peg by sampling more tactile observations. Second, it selects the next action for the robot to sample the next touch (and thus image) which results in maximum uncertainty reduction to minimize the number of interactions during the perception task. We evaluate our method on several part-mating tasks for assembly using a robot equipped with a vision-based tactile sensor. We also show the efficiency of the proposed action selection method against a naive method. See supplementary video at https://www.youtube.com/watch?v=jMVBg_e3gLw .

翻译：人类依靠触觉和触觉感知完成大量灵巧操作任务。我们的触觉感知能在任何交互过程中提供关于接触形态以及物体几何信息的大量数据。受此启发，基于视觉的触觉传感器正被广泛应用于各类机器人感知与控制任务中。本文提出一种利用基于视觉的触觉传感器进行多物体装配的交互式感知方法。具体而言，我们关注零件配合过程中的触觉感知，机器人可利用触觉传感器和基于粒子滤波的反馈机制，逐步优化对可装配对象的估计。为此，我们首先训练一个深度神经网络，利用触觉图像预测任意形状可配合物体之间的概率对应关系。该训练模型被用于设计双重功能的粒子滤波器：第一，在仅观察到孔的部分（或非唯一）特征时，通过采样更多触觉观测逐步优化正确插件的估计；第二，选择机器人的下一步动作以采样下一个触觉信息（从而获取图像），实现最大不确定性缩减，最小化感知任务中的交互次数。我们通过在配备基于视觉触觉传感器的机器人上执行多个零件装配任务来评估该方法，并展示了所提出的动作选择方法相较于朴素方法的效率提升。补充视频见 https://www.youtube.com/watch?v=jMVBg_e3gLw 。