In this contribution, we introduce the concept of Instance Performance Difference (IPD), a metric designed to measure the gap in performance that a robotics perception task experiences when working with real vs. synthetic pictures. By pairing synthetic and real instances in the pictures and evaluating their performance similarity using perception algorithms, IPD provides a targeted metric that closely aligns with the needs of real-world applications. We explain and demonstrate this metric through a rock detection task in lunar terrain images, highlighting the IPD's effectiveness in identifying the most realistic image synthesis method. The metric is thus instrumental in creating synthetic image datasets that perform in perception tasks like real-world photo counterparts. In turn, this supports robust sim-to-real transfer for perception algorithms in real-world robotics applications.

翻译：本文提出了实例性能差异（IPD）的概念，这是一种旨在衡量机器人感知任务在处理真实图像与合成图像时性能差距的指标。通过配对图像中的合成实例与真实实例，并利用感知算法评估其性能相似性，IPD提供了一种紧密贴合实际应用需求的有针对性度量。我们以月球地形图像中的岩石检测任务为例，阐释并论证了这一指标，突显了IPD在识别最逼真图像合成方法方面的有效性。该指标因此有助于创建在感知任务中表现可与真实世界照片相媲美的合成图像数据集。这进而为现实机器人应用中感知算法的稳健模拟到真实迁移提供了支持。