Assistive robots should be able to wash, fold or iron clothes. However, due to the variety, deformability and self-occlusions of clothes, creating general-purpose robot systems for cloth manipulation is challenging. Synthetic data is a promising direction to improve generalization, though its usability is often limited by the sim-to-real gap. To advance the use of synthetic data for cloth manipulation and to enable tasks such as robotic folding, we present a synthetic data pipeline to train keypoint detectors for almost flattened cloth items. To test its performance, we have also collected a real-world dataset. We train detectors for both T-shirts, towels and shorts and obtain an average precision of 64.3%. Fine-tuning on real-world data improves performance to 74.2%. Additional insight is provided by discussing various failure modes of the keypoint detectors and by comparing different approaches to obtain cloth meshes and materials. We also quantify the remaining sim-to-real gap and argue that further improvements to the fidelity of cloth assets will be required to further reduce this gap. The code, dataset and trained models are available online.

翻译：辅助机器人应能够清洗、折叠或熨烫衣物。然而，由于衣物的多样性、可变形性及自遮挡特性，构建通用型机器人布料操作系统具有挑战性。合成数据是提升泛化能力的一个有前景的方向，但其可用性常受到仿真到现实差距的限制。为推进合成数据在布料操作中的应用，并实现机器人折叠等任务，我们提出了一种用于训练近乎平铺布料物品关键点检测器的合成数据流程。为评估其性能，我们还收集了一个真实数据集。我们针对T恤、毛巾和短裤训练了检测器，平均精度达到64.3%。在真实数据上进行微调可将性能提升至74.2%。通过对关键点检测器各类失效模式的讨论，以及比较获取布料网格和材质的不同方法，本文提供了进一步见解。我们还量化了剩余的仿真到现实差距，并指出需进一步提高布料资产的逼真度以缩小该差距。代码、数据集及训练模型均可在线获取。