As robots increasingly coexist with humans, they must navigate complex, dynamic environments rich in visual information and implicit social dynamics, like when to yield or move through crowds. Addressing these challenges requires significant advances in vision-based sensing and a deeper understanding of socio-dynamic factors, particularly in tasks like navigation. To facilitate this, robotics researchers need advanced simulation platforms offering dynamic, photorealistic environments with realistic actors. Unfortunately, most existing simulators fall short, prioritizing geometric accuracy over visual fidelity, and employing unrealistic agents with fixed trajectories and low-quality visuals. To overcome these limitations, we developed a simulator that incorporates three essential elements: (1) photorealistic neural rendering of environments, (2) neurally animated human entities with behavior management, and (3) an ego-centric robotic agent providing multi-sensor output. By utilizing advanced neural rendering techniques in a dual-NeRF simulator, our system produces high-fidelity, photorealistic renderings of both environments and human entities. Additionally, it integrates a state-of-the-art Social Force Model to model dynamic human-human and human-robot interactions, creating the first photorealistic and accessible human-robot simulation system powered by neural rendering.

翻译：随着机器人与人类日益共存，它们必须在复杂动态的环境中导航，这些环境富含视觉信息与隐含的社会动态，例如何时避让或如何在人群中穿行。应对这些挑战需要基于视觉的感知技术取得重大进展，并更深入地理解社会动态因素，尤其在导航等任务中。为促进这一目标，机器人学研究者需要先进的仿真平台，提供具有真实感角色的动态高真实感环境。遗憾的是，现有大多数仿真器存在不足，它们优先考虑几何精度而牺牲视觉保真度，并采用轨迹固定、视觉质量低下的非真实感智能体。为克服这些局限，我们开发了一种融合三个关键要素的仿真器：（1）环境的高真实感神经渲染，（2）具备行为管理的神经动画化人类实体，以及（3）提供多传感器输出的以自我为中心的机器人智能体。通过在双NeRF仿真器中运用先进的神经渲染技术，我们的系统能生成环境与人类实体的高保真、高真实感渲染结果。此外，该系统集成了最先进的社会力模型，以模拟动态的人-人及人-机器人交互，从而创建了首个由神经渲染驱动的高真实感且易于使用的人机交互仿真系统。