Why do Recurrent State Space Models such as PlaNet fail at cloth manipulation tasks? Recent work has attributed this to the blurry prediction of the observation, which makes it difficult to plan directly in the latent space. This paper explores the reasons behind this by applying PlaNet in the pick-and-place fabric-flattening domain. We find that the sharp discontinuity of the transition function on the contour of the fabric makes it difficult to learn an accurate latent dynamic model, causing the MPC planner to produce pick actions slightly outside of the article. By limiting picking space on the cloth mask and training on specially engineered trajectories, our mesh-free PlaNet-ClothPick surpasses visual planning and policy learning methods on principal metrics in simulation, achieving similar performance as state-of-the-art mesh-based planning approaches. Notably, our model exhibits a faster action inference and requires fewer transitional model parameters than the state-of-the-art robotic systems in this domain. Other supplementary materials are available at: https://sites.google.com/view/planet-clothpick.

翻译：为什么PlaNet这类循环状态空间模型在布料操控任务中表现不佳？近期研究将其归因于观测预测的模糊性，导致难以在潜在空间中进行直接规划。本文通过将PlaNet应用于抓取放置式织物展平领域，深入探究了其失败原因。研究发现，过渡函数在织物轮廓处存在急剧不连续性，这阻碍了精确潜在动态模型的学习，使得MPC规划器生成超出织物范围的抓取动作。通过限制布料掩膜上的抓取空间，并基于特殊设计的轨迹进行训练，本文提出的无网格PlaNet-ClothPick在仿真环境的主要指标上超越了视觉规划与策略学习方法，性能接近基于网格的最优规划方案。值得关注的是，我们的模型在动作推理速度上显著提升，且相比该领域最先进的机器人系统，所需的过渡模型参数更少。其他补充材料详见：https://sites.google.com/view/planet-clothpick。