To enable general-purpose robots, we will require the robot to operate daily articulated objects as humans do. Current robot manipulation has heavily relied on using a parallel gripper, which restricts the robot to a limited set of objects. On the other hand, operating with a multi-finger robot hand will allow better approximation to human behavior and enable the robot to operate on diverse articulated objects. To this end, we propose a new benchmark called DexArt, which involves Dexterous manipulation with Articulated objects in a physical simulator. In our benchmark, we define multiple complex manipulation tasks, and the robot hand will need to manipulate diverse articulated objects within each task. Our main focus is to evaluate the generalizability of the learned policy on unseen articulated objects. This is very challenging given the high degrees of freedom of both hands and objects. We use Reinforcement Learning with 3D representation learning to achieve generalization. Through extensive studies, we provide new insights into how 3D representation learning affects decision making in RL with 3D point cloud inputs. More details can be found at https://www.chenbao.tech/dexart/.

翻译：为实现通用型机器人，机器人需要像人类一样操作日常铰接物体。当前机器人操作严重依赖平行夹爪，这限制了机器人可操作的物体种类。而采用多指机器人手部操作能更好模拟人类行为，使机器人能够操作各类铰接物体。为此，我们提出名为DexArt的新型基准测试——在物理仿真器中实现铰接物体的灵巧操作。该基准定义了多个复杂操作任务，机器人手部需在每项任务中操作多样化的铰接物体。核心目标是评估所学策略对未见铰接物体的泛化能力。由于手部与物体均具有高自由度，这一目标极具挑战性。我们采用结合三维表征学习的强化学习实现泛化。通过大量实验，本研究揭示了三维点云输入下三维表征学习如何影响强化学习决策的新见解。更多细节请参见https://www.chenbao.tech/dexart/。