Nowadays, autonomous cars can drive smoothly in ordinary cases, and it is widely recognized that realistic sensor simulation will play a critical role in solving remaining corner cases by simulating them. To this end, we propose an autonomous driving simulator based upon neural radiance fields (NeRFs). Compared with existing works, ours has three notable features: (1) Instance-aware. Our simulator models the foreground instances and background environments separately with independent networks so that the static (e.g., size and appearance) and dynamic (e.g., trajectory) properties of instances can be controlled separately. (2) Modular. Our simulator allows flexible switching between different modern NeRF-related backbones, sampling strategies, input modalities, etc. We expect this modular design to boost academic progress and industrial deployment of NeRF-based autonomous driving simulation. (3) Realistic. Our simulator set new state-of-the-art photo-realism results given the best module selection. Our simulator will be open-sourced while most of our counterparts are not. Project page: https://open-air-sun.github.io/mars/.

翻译：如今，自动驾驶汽车在常规情况下已能平稳行驶，业界普遍认为逼真的传感器模拟将通过模拟剩余边缘场景在其中发挥关键作用。为此，我们提出一种基于神经辐射场（NeRF）的自动驾驶模拟器。与现有工作相比，我们的模拟器具有三个显著特征：（1）实例感知。本模拟器通过独立网络分别对前景实例与背景环境进行建模，从而实现对实例的静态属性（如尺寸、外观）与动态属性（如轨迹）的独立控制。（2）模块化。本模拟器支持在不同现代NeRF相关骨干网络、采样策略、输入模态等之间灵活切换。我们期望这种模块化设计能推动基于NeRF的自动驾驶模拟的学术进步与工业部署。（3）逼真性。在最佳模块选择下，本模拟器在照片级真实感方面达到了新的最先进水平。本模拟器将开源发布，而大多数同类模拟器则未开源。项目主页：https://open-air-sun.github.io/mars/。