Multiple Peg-in-Hole (MPiH) assembly is one of the fundamental tasks in robotic assembly. In the MPiH tasks for large-size parts, it is challenging for a single manipulator to simultaneously align multiple distant pegs and holes, necessitating tightly coupled multi-manipulator systems. For such MPiH tasks using tightly coupled multiple manipulators, we propose a collaborative visual servo control framework that uses only the monocular in-hand cameras of each manipulator to reduce positioning errors. Initially, we train a state classification neural network and a positioning neural network. The former divides the states of the peg and hole in the image into three categories: obscured, separated, and overlapped, while the latter determines the position of the peg and hole in the image. Based on these findings, we propose a method to integrate the visual features of multiple manipulators using virtual forces, which can naturally combine with the cooperative controller of the multi-manipulator system. To generalize our approach to holes of different appearances, we varied the appearance of the holes during the dataset generation process. The results confirm that by considering the appearance of the holes, classification accuracy and positioning precision can be improved. Finally, the results show that our method achieves 100\% success rate in dual-manipulator dual peg-in-hole tasks with a clearance of 0.2 mm, while robust to camera calibration errors.

翻译：多孔装配是机器人装配中的基本任务之一。在大型零件的多孔装配任务中，单个机械臂难以同时对准多个相距较远的轴孔，因此需要采用紧密耦合的多机械臂系统。针对这种使用紧密耦合多机械臂的多孔装配任务，我们提出了一种协作视觉伺服控制框架，该框架仅利用每个机械臂的单目手眼相机来减小定位误差。首先，我们训练了一个状态分类神经网络和一个定位神经网络。前者将图像中轴与孔的状态分为三类：遮挡、分离和重叠，后者则确定轴与孔在图像中的位置。基于这些发现，我们提出了一种利用虚拟力融合多机械臂视觉特征的方法，该方法能够自然地与多机械臂系统的协同控制器相结合。为了使我们的方法能够泛化到不同外观的孔，我们在数据集生成过程中改变了孔的外观。结果证实，通过考虑孔的外观特征，可以提高分类精度和定位精度。最终实验结果表明，在间隙为0.2毫米的双机械臂双孔装配任务中，我们的方法实现了100%的成功率，同时对相机标定误差具有鲁棒性。