Benchmarking provides experimental evidence of the scientific baseline to enhance the progression of fundamental research, which is also applicable to robotics. In this paper, we propose a method to benchmark metrics of robotic manipulation, which addresses the spatial-temporal reasoning skills for robot learning with the jigsaw game. In particular, our approach exploits a simple set of jigsaw pieces by designing a structured protocol, which can be highly customizable according to a wide range of task specifications. Researchers can selectively adopt the proposed protocol to benchmark their research outputs, on a comparable scale in the functional, task, and system-level of details. The purpose is to provide a potential look-up table for learning-based robot manipulation, commonly available in other engineering disciplines, to facilitate the adoption of robotics through calculated, empirical, and systematic experimental evidence.

翻译：基准测试通过提供科学基线的实验证据来促进基础研究进步，这一方法同样适用于机器人领域。本文提出了一种针对机器人操作指标的基准测试方法，通过拼图游戏解决机器人学习中的时空推理能力问题。具体而言，我们的方法利用一套简易拼图模块，通过设计结构化协议实现高度可定制化，能够适应多种任务规范要求。研究人员可根据功能、任务及系统层级细节的可比规模，选择性采用该协议对其研究成果进行基准测试。本研究的目的是为基于学习的机器人操作领域提供一种潜在参考对照体系（该体系在其他工程学科中已普遍存在），从而通过经过计算验证的、经验性的系统实验证据推动机器人技术的实际应用。