Deep Reinforcement learning (DRL) has achieved remarkable success in domains with well-defined reward structures, such as Atari games and locomotion. In contrast, dexterous manipulation lacks general-purpose reward formulations and typically depends on task-specific, handcrafted priors to guide hand-object interactions. We propose Contact Coverage-Guided Exploration (CCGE), a general exploration method designed for general-purpose dexterous manipulation tasks. CCGE represents contact state as the intersection between object surface points and predefined hand keypoints, encouraging dexterous hands to discover diverse and novel contact patterns, namely which fingers contact which object regions. It maintains a contact counter conditioned on discretized object states obtained via learned hash codes, capturing how frequently each finger interacts with different object regions. This counter is leveraged in two complementary ways: (1) to assign a count-based contact coverage reward that promotes exploration of novel contact patterns, and (2) an energy-based reaching reward that guides the agent toward under-explored contact regions. We evaluate CCGE on a diverse set of dexterous manipulation tasks, including cluttered object singulation, constrained object retrieval, in-hand reorientation, and bimanual manipulation. Experimental results show that CCGE substantially improves training efficiency and success rates over existing exploration methods, and that the contact patterns learned with CCGE transfer robustly to real-world robotic systems. Project page is https://contact-coverage-guided-exploration.github.io.

翻译：深度强化学习（DRL）在奖励结构明确的领域（如Atari游戏与运动控制）已取得显著成功。相比之下，灵巧操作任务缺乏通用的奖励函数设计，通常依赖针对特定任务手工设计的先验知识来引导手-物体交互。本文提出接触覆盖引导探索（CCGE），一种专为通用灵巧操作任务设计的探索方法。CCGE将接触状态表征为物体表面点与预定义手部关键点的交集，激励灵巧手发现多样新颖的接触模式（即哪些手指接触哪些物体区域）。该方法通过学习得到的哈希码对物体状态进行离散化，并维护基于此的条件化接触计数器，以记录每个手指与不同物体区域的交互频率。该计数器通过两种互补方式被利用：（1）基于计数的接触覆盖奖励，促进对新颖接触模式的探索；（2）基于能量的接近奖励，引导智能体朝向未充分探索的接触区域。我们在多种灵巧操作任务上评估CCGE，包括密集物体分离、受限物体抓取、手内重定向及双手操作。实验结果表明，相较于现有探索方法，CCGE显著提升了训练效率与成功率，且通过CCGE学习的接触模式能够稳健地迁移至真实机器人系统。项目页面见 https://contact-coverage-guided-exploration.github.io。