We present a new reproducible benchmark for evaluating robot manipulation in the real world, specifically focusing on pick-and-place. Our benchmark uses the YCB objects, a commonly used dataset in the robotics community, to ensure that our results are comparable to other studies. Additionally, the benchmark is designed to be easily reproducible in the real world, making it accessible to researchers and practitioners. We also provide our experimental results and analyzes for model-based and model-free 6D robotic grasping on the benchmark, where representative algorithms are evaluated for object perception, grasping planning, and motion planning. We believe that our benchmark will be a valuable tool for advancing the field of robot manipulation. By providing a standardized evaluation framework, researchers can more easily compare different techniques and algorithms, leading to faster progress in developing robot manipulation methods.

翻译：我们提出了一种新的可复现基准测试，用于评估真实世界中的机器人操作，特别聚焦于拾取与放置任务。该基准测试采用机器人社区常用的YCB物体数据集，以确保我们的结果可与其他研究进行比较。此外，该基准测试被设计为易于在真实世界中复现，便于研究人员和从业者使用。我们还提供了在此基准测试上对基于模型和无模型的6D机器人抓取进行的实验结果与分析，其中对代表性算法的物体感知、抓取规划与运动规划能力进行了评估。我们相信，该基准测试将成为推动机器人操作领域发展的宝贵工具。通过提供标准化的评估框架，研究人员可以更轻松地比较不同技术与算法，从而加速机器人操作方法的研发进展。