Deformable object manipulation is a classical and challenging research area in robotics. Compared with rigid object manipulation, this problem is more complex due to the deformation properties including elastic, plastic, and elastoplastic deformation. In this paper, we describe a new deformable object manipulation method including soft contact simulation, manipulation learning, and sim-to-real transfer. We propose a novel approach utilizing Vision-Based Tactile Sensors (VBTSs) as the end-effector in simulation to produce observations like relative position, squeezed area, and object contour, which are transferable to real robots. For a more realistic contact simulation, a new simulation environment including elastic, plastic, and elastoplastic deformations is created. We utilize RL strategies to train agents in the simulation, and expert demonstrations are applied for challenging tasks. Finally, we build a real experimental platform to complete the sim-to-real transfer and achieve a 90% success rate on difficult tasks such as cylinder and sphere. To test the robustness of our method, we use plasticine of different hardness and sizes to repeat the tasks including cylinder and sphere. The experimental results show superior performances of deformable object manipulation with the proposed method.

翻译：软性物体操作是机器人学中一个经典且富有挑战性的研究领域。与刚性物体操作相比，由于涉及弹性、塑性及弹塑性等变形特性，该问题更为复杂。本文描述了一种包含软接触仿真、操作学习以及仿真到现实迁移的新型软性物体操作方法。我们提出一种创新方法，在仿真环境中将基于视觉的触觉传感器（VBTSs）作为末端执行器，以生成可迁移至真实机器人的观测数据（如相对位置、挤压区域及物体轮廓）。为模拟更真实的接触过程，我们构建了包含弹性、塑性及弹塑性变形的新型仿真环境。通过强化学习策略在仿真中训练智能体，并针对复杂任务引入专家演示。最终搭建真实实验平台完成仿真到现实的迁移，在圆柱体与球体等困难任务中实现90%的成功率。为验证方法鲁棒性，我们采用不同硬度与尺寸的橡皮泥重复上述任务。实验结果表明，所提方法在软性物体操作中展现出优越性能。