In this paper, we propose a real-world benchmark for studying robotic learning in the context of functional manipulation: a robot needs to accomplish complex long-horizon behaviors by composing individual manipulation skills in functionally relevant ways. The core design principles of our Functional Manipulation Benchmark (FMB) emphasize a harmonious balance between complexity and accessibility. Tasks are deliberately scoped to be narrow, ensuring that models and datasets of manageable scale can be utilized effectively to track progress. Simultaneously, they are diverse enough to pose a significant generalization challenge. Furthermore, the benchmark is designed to be easily replicable, encompassing all essential hardware and software components. To achieve this goal, FMB consists of a variety of 3D-printed objects designed for easy and accurate replication by other researchers. The objects are procedurally generated, providing a principled framework to study generalization in a controlled fashion. We focus on fundamental manipulation skills, including grasping, repositioning, and a range of assembly behaviors. The FMB can be used to evaluate methods for acquiring individual skills, as well as methods for combining and ordering such skills to solve complex, multi-stage manipulation tasks. We also offer an imitation learning framework that includes a suite of policies trained to solve the proposed tasks. This enables researchers to utilize our tasks as a versatile toolkit for examining various parts of the pipeline. For example, researchers could propose a better design for a grasping controller and evaluate it in combination with our baseline reorientation and assembly policies as part of a pipeline for solving multi-stage tasks. Our dataset, object CAD files, code, and evaluation videos can be found on our project website: https://functional-manipulation-benchmark.github.io

翻译：本文提出一个用于研究机器人功能性操作学习的真实世界基准：机器人需要通过以功能相关的方式组合个体操作技能来完成复杂的长期行为。我们的功能性操作基准（FMB）核心设计原则强调复杂性与可及性之间的和谐平衡。任务范围经过精心限定，确保可利用规模适中的模型与数据集有效追踪进展；同时任务具备足够多样性以构成显著的泛化挑战。该基准设计易于复现，涵盖所有必要的硬件与软件组件。为实现此目标，FMB包含一系列专为其他研究者便捷精确复现而设计的3D打印物体。这些物体通过程序化生成，为在受控环境下研究泛化问题提供原则性框架。我们聚焦于抓取、重定向及各类装配行为等基础操作技能。FMB既可用于评估个体技能习得方法，也可用于评估组合排序这些技能以解决复杂多阶段操作任务的方法。我们还提供包含整套任务求解策略的模仿学习框架，使研究者能将我们的任务作为多功能工具包来检验流程中各环节。例如，研究者可提出改进的抓取控制器设计，并将其与我们的基线重定向及装配策略结合，作为求解多阶段任务流程的组成部分进行评估。我们的数据集、物体CAD文件、代码及评估视频可在项目网站获取：https://functional-manipulation-benchmark.github.io