Physical manipulation of garments is often crucial when performing fabric-related tasks, such as hanging garments. However, due to the deformable nature of fabrics, these operations remain a significant challenge for robots in household, healthcare, and industrial environments. In this paper, we propose GraphGarment, a novel approach that models garment dynamics based on robot control inputs and applies the learned dynamics model to facilitate garment manipulation tasks such as hanging. Specifically, we use graphs to represent the interactions between the robot end-effector and the garment. GraphGarment uses a graph neural network (GNN) to learn a dynamics model that can predict the next garment state given the current state and input action in simulation. To address the substantial sim-to-real gap, we propose a residual model that compensates for garment state prediction errors, thereby improving real-world performance. The garment dynamics model is then applied to a model-based action sampling strategy, where it is utilized to manipulate the garment to a reference pre-hanging configuration for garment-hanging tasks. We conducted four experiments using six types of garments to validate our approach in both simulation and real-world settings. In simulation experiments, GraphGarment achieves better garment state prediction performance, with a prediction error 0.46 cm lower than the best baseline. Our approach also demonstrates improved performance in the garment-hanging simulation experiment with enhancements of 12%, 24%, and 10%, respectively. Moreover, real-world robot experiments confirm the robustness of sim-to-real transfer, with an error increase of 0.17 cm compared to simulation results. Supplementary material is available at:https://sites.google.com/view/graphgarment.

翻译：在执行与织物相关的任务（例如悬挂衣物）时，对服装进行物理操作通常是至关重要的。然而，由于织物的可变形特性，这些操作对于家庭、医疗和工业环境中的机器人而言仍然是一个重大挑战。本文提出GraphGarment，这是一种新颖的方法，它基于机器人控制输入对服装动力学进行建模，并应用学习到的动力学模型来促进诸如悬挂等服装操作任务。具体而言，我们使用图来表示机器人末端执行器与服装之间的相互作用。GraphGarment使用图神经网络（GNN）来学习一个动力学模型，该模型可以在仿真中给定当前状态和输入动作的情况下预测下一个服装状态。为了解决显著的仿真到现实差距，我们提出了一种残差模型来补偿服装状态预测误差，从而提高现实世界中的性能。随后，将服装动力学模型应用于一种基于模型的动作采样策略中，利用该策略将服装操作至参考的预悬挂配置，以完成衣物悬挂任务。我们使用六种类型的服装进行了四项实验，以在仿真和现实世界环境中验证我们的方法。在仿真实验中，GraphGarment实现了更好的服装状态预测性能，其预测误差比最佳基线低0.46厘米。我们的方法在衣物悬挂仿真实验中也表现出性能提升，分别提高了12%、24%和10%。此外，现实世界机器人实验证实了仿真到现实迁移的鲁棒性，与仿真结果相比误差仅增加0.17厘米。补充材料可在以下网址获取：https://sites.google.com/view/graphgarment。