Garment manipulation is a critical challenge due to the diversity in garment categories, geometries, and deformations. Despite this, humans can effortlessly handle garments, thanks to the dexterity of our hands. However, existing research in the field has struggled to replicate this level of dexterity, primarily hindered by the lack of realistic simulations of dexterous garment manipulation. Therefore, we propose DexGarmentLab, the first environment specifically designed for dexterous (especially bimanual) garment manipulation, which features large-scale high-quality 3D assets for 15 task scenarios, and refines simulation techniques tailored for garment modeling to reduce the sim-to-real gap. Previous data collection typically relies on teleoperation or training expert reinforcement learning (RL) policies, which are labor-intensive and inefficient. In this paper, we leverage garment structural correspondence to automatically generate a dataset with diverse trajectories using only a single expert demonstration, significantly reducing manual intervention. However, even extensive demonstrations cannot cover the infinite states of garments, which necessitates the exploration of new algorithms. To improve generalization across diverse garment shapes and deformations, we propose a Hierarchical gArment-manipuLation pOlicy (HALO). It first identifies transferable affordance points to accurately locate the manipulation area, then generates generalizable trajectories to complete the task. Through extensive experiments and detailed analysis of our method and baseline, we demonstrate that HALO consistently outperforms existing methods, successfully generalizing to previously unseen instances even with significant variations in shape and deformation where others fail. Our project page is available at: https://wayrise.github.io/DexGarmentLab/.

翻译：衣物操作因衣物类别、几何形状与形变的多样性而成为一项关键挑战。尽管如此，人类凭借双手的灵巧性能够轻松处理衣物。然而，该领域现有研究一直难以复现这种灵巧水平，主要受限于缺乏对灵巧衣物操作的真实模拟。为此，我们提出了DexGarmentLab，这是首个专为灵巧（尤其是双手）衣物操作设计的环境，其特点包括为15个任务场景提供大规模高质量3D资产，并针对衣物建模优化了仿真技术以缩小仿真与现实的差距。以往的数据收集通常依赖于遥操作或训练专家强化学习策略，这些方法劳动密集且效率低下。本文中，我们利用衣物结构对应性，仅通过单次专家演示即可自动生成包含多样化轨迹的数据集，显著减少了人工干预。然而，即使大量演示也无法覆盖衣物的无限状态，这促使我们探索新算法。为提升对不同衣物形状和形变的泛化能力，我们提出了一种分层式衣物操作策略（HALO）。该策略首先识别可迁移的示能点以精确定位操作区域，随后生成可泛化的轨迹以完成任务。通过对我们的方法与基线进行大量实验和详细分析，我们证明HALO始终优于现有方法，即使在形状和形变存在显著差异、其他方法失效的情况下，也能成功泛化到先前未见过的实例。项目页面位于：https://wayrise.github.io/DexGarmentLab/。