In this paper, we propose a real-world benchmark for studying robotic learning in the context of functional manipulation: a robot needs to accomplish complex long-horizon behaviors by composing individual manipulation skills in functionally relevant ways. The core design principles of our Functional Manipulation Benchmark (FMB) emphasize a harmonious balance between complexity and accessibility. Tasks are deliberately scoped to be narrow, ensuring that models and datasets of manageable scale can be utilized effectively to track progress. Simultaneously, they are diverse enough to pose a significant generalization challenge. Furthermore, the benchmark is designed to be easily replicable, encompassing all essential hardware and software components. To achieve this goal, FMB consists of a variety of 3D-printed objects designed for easy and accurate replication by other researchers. The objects are procedurally generated, providing a principled framework to study generalization in a controlled fashion. We focus on fundamental manipulation skills, including grasping, repositioning, and a range of assembly behaviors. The FMB can be used to evaluate methods for acquiring individual skills, as well as methods for combining and ordering such skills to solve complex, multi-stage manipulation tasks. We also offer an imitation learning framework that includes a suite of policies trained to solve the proposed tasks. This enables researchers to utilize our tasks as a versatile toolkit for examining various parts of the pipeline. For example, researchers could propose a better design for a grasping controller and evaluate it in combination with our baseline reorientation and assembly policies as part of a pipeline for solving multi-stage tasks. Our dataset, object CAD files, code, and evaluation videos can be found on our project website: https://functional-manipulation-benchmark.github.io

翻译：摘要：本文提出了一个用于研究机器人学习在功能操作情境下的真实世界基准：机器人需要以功能相关的方式组合单个操作技能，完成复杂的长期行为。我们提出的功能操作基准（FMB）核心设计原则强调复杂性与可访问性之间的和谐平衡。任务被刻意限定在狭窄范围内，确保可有效利用规模适中的模型和数据集跟踪进展，同时其多样性足以构成显著的泛化挑战。此外，该基准设计易于复现，包含所有必要的硬件和软件组件。为实现这一目标，FMB包含多种3D打印物体，便于其他研究者精准复现。这些物体通过程序化生成，为在受控条件下研究泛化提供了原则性框架。我们聚焦于基础操作技能，包括抓取、重新定位及一系列装配行为。FMB可用于评估单项技能的获取方法，以及组合、排序这些技能以解决复杂多阶段操作任务的方法。我们还提供了一个模仿学习框架，包含一组经过训练的策略以解决所提出的任务，使研究者能够将我们的任务作为用于检验流水线各环节的多功能工具包。例如，研究者可以提出更好的抓取控制器设计方案，并将其与我们基线重定向和装配策略组合评估，作为解决多阶段任务的流水线组成部分。我们的数据集、物体CAD文件、代码及评估视频可在项目网站获取：https://functional-manipulation-benchmark.github.io