Extensive research has been conducted on assessing grasp stability, a crucial prerequisite for achieving optimal grasping strategies, including the minimum force grasping policy. However, existing works employ basic feature-level fusion techniques to combine visual and tactile modalities, resulting in the inadequate utilization of complementary information and the inability to model interactions between unimodal features. This work proposes an attention-guided cross-modality fusion architecture to comprehensively integrate visual and tactile features. This model mainly comprises convolutional neural networks (CNNs), self-attention, and cross-attention mechanisms. In addition, most existing methods collect datasets from real-world systems, which is time-consuming and high-cost, and the datasets collected are comparatively limited in size. This work establishes a robotic grasping system through physics simulation to collect a multimodal dataset. To address the sim-to-real transfer gap, we propose a migration strategy encompassing domain randomization and domain adaptation techniques. The experimental results demonstrate that the proposed fusion framework achieves markedly enhanced prediction performance (approximately 10%) compared to other baselines. Moreover, our findings suggest that the trained model can be reliably transferred to real robotic systems, indicating its potential to address real-world challenges.

翻译：针对抓取稳定性评估这一实现最优抓取策略（包括最小力抓取策略）的关键前置条件，已有大量研究展开。然而现有工作采用基础特征级融合技术结合视觉与触觉模态，导致互补信息利用不足，且无法建模单模态特征间的交互关系。本文提出一种基于注意力引导的跨模态融合架构，以全面整合视觉与触觉特征。该模型主要由卷积神经网络（CNN）、自注意力机制和交叉注意力机制构成。此外，现有方法多从真实系统采集数据集，存在耗时久、成本高的问题，且所采集数据集规模相对有限。本研究通过物理仿真构建机器人抓取系统以采集多模态数据集。针对仿真到真实环境的迁移差距，我们提出包含域随机化和域自适应技术的迁移策略。实验结果表明，与其他基线方法相比，所提出的融合框架预测性能显著提升（约10%）。此外，研究结果表明训练模型可可靠迁移至真实机器人系统，展现了其解决实际问题的潜力。