Assistive robots should be able to wash, fold or iron clothes. However, due to the variety, deformability and self-occlusions of clothes, creating robot systems for cloth manipulation is challenging. Synthetic data is a promising direction to improve generalization, but the sim-to-real gap limits its effectiveness. To advance the use of synthetic data for cloth manipulation tasks such as robotic folding, we present a synthetic data pipeline to train keypoint detectors for almost-flattened cloth items. To evaluate its performance, we have also collected a real-world dataset. We train detectors for both T-shirts, towels and shorts and obtain an average precision of 64% and an average keypoint distance of 18 pixels. Fine-tuning on real-world data improves performance to 74% mAP and an average distance of only 9 pixels. Furthermore, we describe failure modes of the keypoint detectors and compare different approaches to obtain cloth meshes and materials. We also quantify the remaining sim-to-real gap and argue that further improvements to the fidelity of cloth assets will be required to further reduce this gap. The code, dataset and trained models are available

翻译：辅助机器人应能够完成洗衣、折叠或熨烫衣物等任务。然而，由于衣物的多样性、可变形性以及自遮挡特性，构建用于布料操作的机器人系统颇具挑战性。合成数据是提升泛化能力的一种有前景的方向，但仿真到现实的差距限制了其有效性。为推进合成数据在机器人折叠等布料操作任务中的应用，我们提出了一种用于训练近乎平整衣物关键点检测器的合成数据流水线。为评估其性能，我们还收集了真实世界数据集。我们对T恤、毛巾和短裤分别训练了检测器，获得了64%的平均精度和18像素的平均关键点距离。在真实世界数据上进行微调后，性能提升至74%的平均精度均值（mAP），平均距离仅9像素。此外，我们描述了关键点检测器的失效模式，并比较了获取布料网格和材质的不同方法。我们还量化了剩余的仿真到现实差距，并认为需要进一步提高布料资产的保真度以缩小这一差距。代码、数据集和训练后的模型均已公开。