Simulation can and should play a critical role in the development and testing of algorithms for autonomous agents. What might reduce its impact is the ``sim2real'' gap -- the algorithm response differs between operation in simulated versus real-world environments. This paper introduces an approach to evaluate this gap, focusing on the accuracy of sensor simulation -- specifically IMU and GPS -- in velocity estimation tasks for autonomous agents. Using a scaled autonomous vehicle, we conduct 40 real-world experiments across diverse environments then replicate the experiments in simulation with five distinct sensor noise models. We note that direct comparison of raw simulation and real sensor data fails to quantify the sim2real gap for robotics applications. We demonstrate that by using a state of the art state-estimation package as a ``judge'', and by evaluating the performance of this state-estimator in both real and simulated scenarios, we can isolate the sim2real discrepancies stemming from sensor simulations alone. The dataset generated is open-source and publicly available for unfettered use.

翻译：仿真可以在自主智能体算法的开发和测试中发挥关键作用。然而，"模拟到现实"的差距——算法在模拟环境与真实环境中的响应差异——可能削弱其影响力。本文提出一种评估这一差距的方法，聚焦于传感器仿真精度（特别是IMU和GPS）在自主智能体速度估计任务中的表现。我们使用一辆缩比自主车辆，在多种环境中进行了40次真实场景实验，随后在仿真中采用五种不同的传感器噪声模型复现这些实验。我们注意到，直接比较原始仿真数据与真实传感器数据无法量化机器人应用中模拟与现实的差距。通过采用最先进的状态估计包作为"评判者"，并评估该状态估计器在真实与模拟场景下的性能表现，我们得以分离出单纯由传感器仿真引起的模拟-现实差异。本文生成的数据集已开源发布，可供自由使用。