Sequentially grasping multiple objects with multi-fingered hands is common in daily life, where humans can fully leverage the dexterity of their hands to enclose multiple objects. However, the diversity of object geometries and the complex contact interactions required for high-DOF hands to grasp one object while enclosing another make sequential multi-object grasping challenging for robots. In this paper, we propose SeqMultiGrasp, a system for sequentially grasping objects with a four-fingered Allegro Hand. We focus on sequentially grasping two objects, ensuring that the hand fully encloses one object before lifting it and then grasps the second object without dropping the first. Our system first synthesizes single-object grasp candidates, where each grasp is constrained to use only a subset of the hand's links. These grasps are then validated in a physics simulator to ensure stability and feasibility. Next, we merge the validated single-object grasp poses to construct multi-object grasp configurations. For real-world deployment, we train a diffusion model conditioned on point clouds to propose grasp poses, followed by a heuristic-based execution strategy. We test our system using $8 \times 8$ object combinations in simulation and $6 \times 3$ object combinations in real. Our diffusion-based grasp model obtains an average success rate of 65.8% over 1600 simulation trials and 56.7% over 90 real-world trials, suggesting that it is a promising approach for sequential multi-object grasping with multi-fingered hands. Supplementary material is available on our project website: https://hesic73.github.io/SeqMultiGrasp.

翻译：使用多指手序列化抓取多个物体在日常生活中十分常见，人类能够充分利用其手部的灵巧性来包裹多个物体。然而，物体几何形状的多样性，以及高自由度手部在抓取一个物体的同时包裹另一个物体所需的复杂接触交互，使得序列化多物体抓取对机器人而言具有挑战性。在本文中，我们提出了SeqMultiGrasp系统，用于使用四指Allegro手序列化抓取物体。我们专注于序列化抓取两个物体，确保手部在提起第一个物体前将其完全包裹，然后在抓取第二个物体时不掉落第一个。我们的系统首先生成单物体抓取候选姿态，其中每个抓取被约束为仅使用手部连杆的一个子集。随后在物理模拟器中验证这些抓取姿态以确保其稳定性和可行性。接下来，我们合并已验证的单物体抓取姿态以构建多物体抓取配置。为了进行真实世界部署，我们训练了一个以点云为条件的扩散模型来提出抓取姿态，随后采用基于启发式的执行策略。我们在模拟环境中使用 $8 \times 8$ 种物体组合，在真实环境中使用 $6 \times 3$ 种物体组合测试了我们的系统。我们基于扩散的抓取模型在1600次模拟试验中获得了65.8%的平均成功率，在90次真实世界试验中获得了56.7%的平均成功率，这表明它是实现多指手序列化多物体抓取的一种有前景的方法。补充材料可在我们的项目网站上获取：https://hesic73.github.io/SeqMultiGrasp。