To substantially enhance robot intelligence, there is a pressing need to develop a large model that enables general-purpose robots to proficiently undertake a broad spectrum of manipulation tasks, akin to the versatile task-planning ability exhibited by LLMs. The vast diversity in objects, robots, and manipulation tasks presents huge challenges. Our work introduces a comprehensive framework to develop a foundation model for general robotic manipulation that formalizes a manipulation task as contact synthesis. Specifically, our model takes as input object and robot manipulator point clouds, object physical attributes, target motions, and manipulation region masks. It outputs contact points on the object and associated contact forces or post-contact motions for robots to achieve the desired manipulation task. We perform extensive experiments both in the simulation and real-world settings, manipulating articulated rigid objects, rigid objects, and deformable objects that vary in dimensionality, ranging from one-dimensional objects like ropes to two-dimensional objects like cloth and extending to three-dimensional objects such as plasticine. Our model achieves average success rates of around 90\%. Supplementary materials and videos are available on our project website at https://manifoundationmodel.github.io/.

翻译：为显著提升机器人智能水平，亟需开发一种能够使通用机器人熟练执行广泛操作任务的大型模型，类似于大型语言模型（LLM）所展现的通用任务规划能力。物体、机器人及操作任务的巨大多样性带来了严峻挑战。本研究提出一套综合框架，用于构建面向通用机器人操作的基座模型，该模型将操作任务形式化为接触合成过程。具体而言，模型以物体与机器人操作器点云、物体物理属性、目标运动及操作区域掩码为输入，输出物体上的接触点及相应的接触力或接触后运动，以使机器人完成指定操作任务。我们在仿真和真实场景中开展了广泛实验，操控的对象涵盖铰接刚体、刚体及可变形体，其维度从绳状一维物体、布料类二维物体延伸至橡皮泥类三维物体。模型平均成功率达到约90%。补充材料及演示视频请参见项目网站:https://manifoundationmodel.github.io/。