As one of the emerging challenges in Automated Machine Learning, the Hardware-aware Neural Architecture Search (HW-NAS) tasks can be treated as black-box multi-objective optimization problems (MOPs). An important application of HW-NAS is real-time semantic segmentation, which plays a pivotal role in autonomous driving scenarios. The HW-NAS for real-time semantic segmentation inherently needs to balance multiple optimization objectives, including model accuracy, inference speed, and hardware-specific considerations. Despite its importance, benchmarks have yet to be developed to frame such a challenging task as multi-objective optimization. To bridge the gap, we introduce a tailored streamline to transform the task of HW-NAS for real-time semantic segmentation into standard MOPs. Building upon the streamline, we present a benchmark test suite, CitySeg/MOP, comprising fifteen MOPs derived from the Cityscapes dataset. The CitySeg/MOP test suite is integrated into the EvoXBench platform to provide seamless interfaces with various programming languages (e.g., Python and MATLAB) for instant fitness evaluations. We comprehensively assessed the CitySeg/MOP test suite on various multi-objective evolutionary algorithms, showcasing its versatility and practicality. Source codes are available at https://github.com/EMI-Group/evoxbench.

翻译：作为自动化机器学习中的新兴挑战之一，面向硬件的神经架构搜索（HW-NAS）任务可视为黑箱多目标优化问题（MOPs）。HW-NAS的重要应用之一是实时语义分割，该技术在自动驾驶场景中具有关键作用。面向实时语义分割的HW-NAS本质上需要平衡多个优化目标，包括模型精度、推理速度及特定硬件考量。尽管其重要性显著，但目前尚未有基准测试能够将该挑战性任务构建为多目标优化问题。为弥合这一空白，我们引入了一套定制化流程，将面向实时语义分割的HW-NAS任务转化为标准MOPs。基于该流程，我们提出了一个基准测试套件CitySeg/MOP，包含源自Cityscapes数据集的十五个MOPs。CitySeg/MOP测试套件已集成至EvoXBench平台，可提供与多种编程语言（如Python和MATLAB）的无缝接口，用于即时适应度评估。我们通过多种多目标进化算法全面评估了CitySeg/MOP测试套件，验证了其通用性与实用性。源代码见https://github.com/EMI-Group/evoxbench。