Neurosim is a fast, real-time, high-performance library for simulating sensors such as dynamic vision sensors, RGB cameras, depth sensors, and inertial sensors. It can also simulate agile dynamics of multi-rotor vehicles in complex and dynamic environments. Neurosim can achieve frame rates as high as ~2700 FPS on a desktop GPU. Neurosim integrates with a ZeroMQ-based communication library called Cortex to facilitate seamless integration with machine learning and robotics workflows. Cortex provides a high-throughput, low-latency message-passing system for Python and C++ applications, with native support for NumPy arrays and PyTorch tensors. This paper discusses the design philosophy behind Neurosim and Cortex. It demonstrates how they can be used to (i) train neuromorphic perception and control algorithms, e.g., using self-supervised learning on time-synchronized multi-modal data, and (ii) test real-time implementations of these algorithms in closed-loop. Neurosim and Cortex are available at https://github.com/grasp-lyrl/neurosim .

翻译：Neurosim 是一个快速、实时、高性能的库，用于仿真动态视觉传感器、RGB相机、深度传感器和惯性传感器等传感器。它还能在复杂动态环境中仿真多旋翼飞行器的敏捷动力学。Neurosim 在桌面 GPU 上可实现高达约 2700 FPS 的帧率。Neurosim 集成了一个基于 ZeroMQ 的通信库 Cortex，以促进与机器学习和机器人工作流的无缝集成。Cortex 为 Python 和 C++ 应用程序提供了一个高吞吐、低延迟的消息传递系统，并原生支持 NumPy 数组和 PyTorch 张量。本文讨论了 Neurosim 和 Cortex 背后的设计理念，并展示了如何利用它们来（i）训练神经形态感知与控制算法，例如在时间同步的多模态数据上使用自监督学习，以及（ii）在闭环中测试这些算法的实时实现。Neurosim 和 Cortex 可在 https://github.com/grasp-lyrl/neurosim 获取。