Autonomous vehicles demand high accuracy and robustness of perception algorithms. To develop efficient and scalable perception algorithms, the maximum information should be extracted from the available sensor data. In this work, we present our concept for an end-to-end perception architecture, named DeepSTEP. The deep learning-based architecture processes raw sensor data from the camera, LiDAR, and RaDAR, and combines the extracted data in a deep fusion network. The output of this deep fusion network is a shared feature space, which is used by perception head networks to fulfill several perception tasks, such as object detection or local mapping. DeepSTEP incorporates multiple ideas to advance state of the art: First, combining detection and localization into a single pipeline allows for efficient processing to reduce computational overhead and further improves overall performance. Second, the architecture leverages the temporal domain by using a self-attention mechanism that focuses on the most important features. We believe that our concept of DeepSTEP will advance the development of end-to-end perception systems. The network will be deployed on our research vehicle, which will be used as a platform for data collection, real-world testing, and validation. In conclusion, DeepSTEP represents a significant advancement in the field of perception for autonomous vehicles. The architecture's end-to-end design, time-aware attention mechanism, and integration of multiple perception tasks make it a promising solution for real-world deployment. This research is a work in progress and presents the first concept of establishing a novel perception pipeline.

翻译：自动驾驶对感知算法的高精度与鲁棒性有着严格要求。为开发高效且可扩展的感知算法，需从可用的传感器数据中提取最大信息。本研究提出了一种名为DeepSTEP的端到端感知架构概念。该基于深度学习的架构处理来自摄像头、激光雷达和雷达的原始传感器数据，并通过深度融合网络整合所提取的数据。深度融合网络的输出是一个共享特征空间，感知头网络利用该空间实现多种感知任务，如目标检测或局部建图。DeepSTEP融合了多项创新以推动技术前沿：首先，将检测与定位整合至单一流程中，可实现高效处理、降低计算开销并进一步提升整体性能；其次，该架构利用自注意力机制聚焦最关键特征，从而发挥时序信息的优势。我们相信DeepSTEP的概念将推动端到端感知系统的发展。该网络将部署于研究车辆，作为数据采集、实车测试与验证的平台。总之，DeepSTEP标志着自动驾驶感知领域的重要进展。其端到端设计、时序感知注意力机制以及多感知任务集成，使其成为面向实际部署的可行方案。本研究为进行中工作，首次提出了构建新型感知流程的概念。