Pareto front profiling in multi-objective optimization (MOO), i.e. finding a diverse set of Pareto optimal solutions, is challenging, especially with expensive objectives like neural network training. Typically, in MOO neural architecture search (NAS), we aim to balance performance and hardware metrics across devices. Prior NAS approaches simplify this task by incorporating hardware constraints into the objective function, but profiling the Pareto front necessitates a search for each constraint. In this work, we propose a novel NAS algorithm that encodes user preferences for the trade-off between performance and hardware metrics, and yields representative and diverse architectures across multiple devices in just one search run. To this end, we parameterize the joint architectural distribution across devices and multiple objectives via a hypernetwork that can be conditioned on hardware features and preference vectors, enabling zero-shot transferability to new devices. Extensive experiments with up to 19 hardware devices and 3 objectives showcase the effectiveness and scalability of our method. Finally, we show that, without additional costs, our method outperforms existing MOO NAS methods across qualitatively different search spaces and datasets, including MobileNetV3 on ImageNet-1k and a Transformer space on machine translation.

翻译：在多目标优化中的帕累托前沿刻画（即寻找一组多样化的帕累托最优解）具有挑战性，尤其当目标函数计算代价高昂（如神经网络训练）时。通常，在多目标神经架构搜索中，我们旨在平衡性能指标与跨设备的硬件指标。现有神经架构搜索方法通过将硬件约束融入目标函数来简化该任务，但帕累托前沿的刻画需要对每个约束进行单独搜索。本文提出一种新型神经架构搜索算法，该算法编码用户对性能与硬件指标间权衡的偏好，并能在单次搜索中生成跨多个设备的代表性且多样化的架构。为此，我们通过一个可基于硬件特征和偏好向量进行条件化的超网络，对跨设备与多目标的联合架构分布进行参数化，从而实现对新型设备的零样本迁移能力。在多达19个硬件设备和3个目标上的大量实验展示了我们方法的有效性与可扩展性。最后，我们证明在无需额外成本的情况下，我们的方法在多个搜索空间和数据集上（包括ImageNet-1k上的MobileNetV3和机器翻译中的Transformer空间）均优于现有Moo神经架构搜索方法。