Multiple peg-in-hole assembly is one of the fundamental tasks in robotic assembly. In the multiple peg-in-hole task for large-sized parts, it is challenging for a single manipulator to simultaneously align multiple distant pegs and holes, necessitating tightly coupled multi-manipulator systems. For such Multi-manipulator Multiple Peg-in-Hole (MMPiH) tasks, we proposes a collaborative visual servo control framework that uses only the monocular in-hand cameras of each manipulator to reduce positioning errors. Initially, we train a state classification neural network and a positioning neural network. The former is used to divide the states of peg and hole in the image into three categories: obscured, separated and overlapped, while the latter determines the position of the peg and hole in the image. Based on these findings, we propose a method to integrate the visual features of multiple manipulators using virtual forces, which can naturally combine with the cooperative controller of the multi-manipulator system. To generalize our approach to holes of different appearances, we varied the appearance of the holes during the dataset generation process. The results confirm that by considering the appearance of the holes, classification accuracy and positioning precision can be improved. Finally, the results show that our method achieves an 85% success rate in dual-manipulator dual peg-in-hole tasks with a clearance of 0.2 mm.

翻译：多孔轴装配是机器人装配中的基础任务之一。针对大型部件的多孔轴装配任务，单一机械臂难以同时对准多个相距较远的轴与孔，因此需要采用紧密耦合的多机械臂系统。针对此类多机械臂多孔轴装配任务，本研究提出一种协作视觉伺服控制框架，该框架仅利用各机械臂的单目手部相机来减小定位误差。首先，我们训练了一个状态分类神经网络和一个定位神经网络：前者用于将图像中轴与孔的状态划分为三类（遮挡、分离、重叠），后者则用于确定图像中轴与孔的位置。基于这些发现，我们提出了一种利用虚拟力融合多机械臂视觉特征的方法，该方法能够自然地与多机械臂系统的协同控制器相结合。为使该方法能泛化至不同外观的孔结构，我们在数据集生成过程中对孔的外观进行了多样化处理。实验结果证实，通过考虑孔的外观特征，分类精度与定位准确度均可得到提升。最终实验结果表明，在间隙为0.2毫米的双机械臂双孔轴装配任务中，本方法取得了85%的成功率。