Garments are important to humans. A visual system that can estimate and track the complete garment pose can be useful for many downstream tasks and real-world applications. In this work, we present a complete package to address the category-level garment pose tracking task: (1) A recording system VR-Garment, with which users can manipulate virtual garment models in simulation through a VR interface. (2) A large-scale dataset VR-Folding, with complex garment pose configurations in manipulation like flattening and folding. (3) An end-to-end online tracking framework GarmentTracking, which predicts complete garment pose both in canonical space and task space given a point cloud sequence. Extensive experiments demonstrate that the proposed GarmentTracking achieves great performance even when the garment has large non-rigid deformation. It outperforms the baseline approach on both speed and accuracy. We hope our proposed solution can serve as a platform for future research. Codes and datasets are available in https://garment-tracking.robotflow.ai.

翻译：服装对人类至关重要。能够估计和追踪完整服装姿态的视觉系统，可用于诸多下游任务及实际应用。本文提出了一套完整的方案，用于解决类别级服装姿态追踪任务：（1）一套记录系统VR-Garment，用户可通过VR界面在仿真环境中操控虚拟服装模型；（2）大规模数据集VR-Folding，包含展平、折叠等操作中的复杂服装姿态配置；（3）端到端在线追踪框架GarmentTracking，该框架可基于点云序列预测服装在规范空间与任务空间中的完整姿态。大量实验表明，即使在服装发生大幅非刚性形变的情况下，所提出的GarmentTracking仍能展现卓越性能。该方法在速度与精度上均优于基线方案。我们期望所提方案能为未来研究提供平台。代码与数据集可从https://garment-tracking.robotflow.ai获取。