While humans intuitively manipulate garments and other textile items swiftly and accurately, it is a significant challenge for robots. A factor crucial to human performance is the ability to imagine, a priori, the intended result of the manipulation intents and hence develop predictions on the garment pose. That ability allows us to plan from highly obstructed states, adapt our plans as we collect more information and react swiftly to unforeseen circumstances. Conversely, robots struggle to establish such intuitions and form tight links between plans and observations. We can partly attribute this to the high cost of obtaining densely labelled data for textile manipulation, both in quality and quantity. The problem of data collection is a long-standing issue in data-based approaches to garment manipulation. As of today, generating high-quality and labelled garment manipulation data is mainly attempted through advanced data capture procedures that create simplified state estimations from real-world observations. However, this work proposes a novel approach to the problem by generating real-world observations from object states. To achieve this, we present GARField (Garment Attached Radiance Field), the first differentiable rendering architecture, to our knowledge, for data generation from simulated states stored as triangle meshes. Code is available on https://ddonatien.github.io/garfield-website/

翻译：尽管人类能够直观、快速且准确地操作服装及其他纺织品，这对机器人而言却构成了一项重大挑战。影响人类表现的一个关键因素在于能够先验地想象操作意图的预期结果，从而预测服装的姿态。这种能力使我们能够从高度遮挡的状态进行规划，在收集更多信息时调整计划，并对意外情况作出快速反应。相比之下，机器人难以建立此类直觉，并在规划与观测之间形成紧密联系。部分原因可归结为获取高质量、大规模的纺织品操作密集标注数据成本高昂。数据收集问题一直是基于数据的服装操作方法中长期存在的难题。目前，生成高质量、带标注的服装操作数据主要通过先进的数据采集流程实现，这些流程从真实世界观测中创建简化的状态估计。然而，本研究提出了一种解决该问题的新颖方法：从物体状态生成真实世界观测。为实现这一目标，我们提出了GARField（服装附着辐射场），据我们所知，这是首个用于从存储为三角形网格的仿真状态生成数据的可微分渲染架构。代码发布于 https://ddonatien.github.io/garfield-website/