Effective planning of long-horizon deformable object manipulation requires suitable abstractions at both the spatial and temporal levels. Previous methods typically either focus on short-horizon tasks or make strong assumptions that full-state information is available, which prevents their use on deformable objects. In this paper, we propose PlAnning with Spatial-Temporal Abstraction (PASTA), which incorporates both spatial abstraction (reasoning about objects and their relations to each other) and temporal abstraction (reasoning over skills instead of low-level actions). Our framework maps high-dimension 3D observations such as point clouds into a set of latent vectors and plans over skill sequences on top of the latent set representation. We show that our method can effectively perform challenging sequential deformable object manipulation tasks in the real world, which require combining multiple tool-use skills such as cutting with a knife, pushing with a pusher, and spreading the dough with a roller.

翻译：长时域可变形物体操作的有效规划需要在空间和时间层面建立合适的抽象。现有方法通常局限于短时域任务，或强假设全状态信息可用，这限制了它们在可变形物体上的应用。本文提出基于时空抽象的规划框架（PASTA），该框架融合空间抽象（推理物体及其相互关系）与时间抽象（基于技能而非底层动作进行推理）。我们的方法将点云等高维3D观测映射为潜在向量集合，并在该潜在集合表示之上规划技能序列。实验表明，该方法能够在真实世界中有效执行具有挑战性的顺序可变形物体操作任务，这些任务需要组合多种工具使用技能，如用刀切割、用推杆推动以及用擀面杖铺平面团。