Roadside perception systems are increasingly crucial in enhancing traffic safety and facilitating cooperative driving for autonomous vehicles. Despite rapid technological advancements, a major challenge persists for this newly arising field: the absence of standardized evaluation methods and benchmarks for these systems. This limitation hampers the ability to effectively assess and compare the performance of different systems, thus constraining progress in this vital field. This paper introduces a comprehensive evaluation methodology specifically designed to assess the performance of roadside perception systems. Our methodology encompasses measurement techniques, metric selection, and experimental trial design, all grounded in real-world field testing to ensure the practical applicability of our approach. We applied our methodology in Mcity\footnote{\url{https://mcity.umich.edu/}}, a controlled testing environment, to evaluate various off-the-shelf perception systems. This approach allowed for an in-depth comparative analysis of their performance in realistic scenarios, offering key insights into their respective strengths and limitations. The findings of this study are poised to inform the development of industry-standard benchmarks and evaluation methods, thereby enhancing the effectiveness of roadside perception system development and deployment for autonomous vehicles. We anticipate that this paper will stimulate essential discourse on standardizing evaluation methods for roadside perception systems, thus pushing the frontiers of this technology. Furthermore, our results offer both academia and industry a comprehensive understanding of the capabilities of contemporary infrastructure-based perception systems.

翻译：路侧感知系统在提升交通安全及促进自动驾驶车辆协同驾驶方面日益关键。尽管技术快速进步，这一新兴领域仍面临重大挑战：缺乏标准化的评估方法及基准。这一局限阻碍了有效评估与比较不同系统性能的能力，从而制约了这一重要领域的发展。本文提出一种专为评估路侧感知系统性能而设计的综合评估方法。该方法涵盖测量技术、指标选择及实验方案设计，均基于真实世界实地测试以确保其实用适用性。我们在受控测试环境Mcity\footnote{\url{https://mcity.umich.edu/}}中应用所提出的方法，评估了多种现成感知系统。这一方法使得在真实场景中对它们进行深入的比较性能分析成为可能，揭示了各自优势与局限的关键洞察。本研究结果有望为行业标准基准与评估方法的制定提供参考，从而提升自动驾驶车辆路侧感知系统开发与部署的有效性。我们预期本文将激发关于标准化路侧感知系统评估方法的必要讨论，进而推动该技术的前沿发展。此外，我们的研究成果为学术界与工业界提供了对当代基础设施感知系统能力的全面理解。