Can robots mold soft plastic materials by shaping depth images? The short answer is no: current day robots can't. In this article, we address the problem of shaping plastic material with an anthropomorphic arm/hand robot, which observes the material with a fixed depth camera. Robots capable of molding could assist humans in many tasks, such as cooking, scooping or gardening. Yet, the problem is complex, due to its high-dimensionality at both perception and control levels. To address it, we design three alternative data-based methods for predicting the effect of robot actions on the material. Then, the robot can plan the sequence of actions and their positions, to mold the material into a desired shape. To make the prediction problem tractable, we rely on two original ideas. First, we prove that under reasonable assumptions, the shaping problem can be mapped from point cloud to depth image space, with many benefits (simpler processing, no need for registration, lower computation time and memory requirements). Second, we design a novel, simple metric for quickly measuring the distance between two depth images. The metric is based on the inherent point cloud representation of depth images, which enables direct and consistent comparison of image pairs through a non-uniform scaling approach, and therefore opens promising perspectives for designing \textit{depth image -- based} robot controllers. We assess our approach in a series of unprecedented experiments, where a robotic arm/hand molds flour from initial to final shapes, either with its own dataset, or by transfer learning from a human dataset. We conclude the article by discussing the limitations of our framework and those of current day hardware, which make human-like robot molding a challenging open research problem.

翻译：机器人能否通过塑形深度图像来模塑软塑料材料？简短的回答是：不能。当前机器人尚不具备这一能力。本文探讨了使用拟人化手臂/手部机器人模塑塑料材料的问题，该机器人通过固定深度相机观察材料。具备模塑能力的机器人可在烹饪、舀取或园艺等多项任务中辅助人类。然而，这一问题因感知与控制层面均具有高维特性而十分复杂。为解决该问题，我们设计了三种基于数据的方法，用于预测机器人动作对材料的影响。随后，机器人可规划动作序列及其位置，将材料塑形为目标形态。为使预测问题易于处理，我们基于两个原创思路。首先，我们证明在合理假设下，塑形问题可从点云空间映射至深度图像空间，并带来诸多优势（简化处理、无需配准、计算时间及内存需求更低）。其次，我们设计了一种新颖的简单度量方法，用于快速计算两幅深度图像间的距离。该度量基于深度图像固有的点云表示，通过非均匀缩放方法实现图像对的直接一致比较，从而为设计基于深度图像的机器人控制器开辟了前景。我们通过一系列前所未有的实验评估了该方法——实验中使用机器人手臂/手部将面粉从初始形态模塑至最终形态，或基于其自有数据集，或通过人类数据集的迁移学习实现。最后，我们讨论了该框架及当前硬件的局限性，这些因素使类人机器人模塑成为一项具有挑战性的开放性研究问题。