Many hardware-aware neural architecture search (NAS) methods have been developed to optimize the topology of neural networks (NN) with the joint objectives of higher accuracy and lower latency. Recently, both accuracy and latency predictors have been used in NAS with great success, achieving high sample efficiency and accurate modeling of hardware (HW) device latency respectively. However, a new accuracy predictor needs to be trained for every new NAS search space or NN task, and a new latency predictor needs to be additionally trained for every new HW device. In this paper, we explore methods to enable multi-task, multi-search-space, and multi-HW adaptation of accuracy and latency predictors to reduce the cost of NAS. We introduce a novel search-space independent NN encoding based on zero-cost proxies that achieves sample-efficient prediction on multiple tasks and NAS search spaces, improving the end-to-end sample efficiency of latency and accuracy predictors by over an order of magnitude in multiple scenarios. For example, our NN encoding enables multi-search-space transfer of latency predictors from NASBench-201 to FBNet (and vice-versa) in under 85 HW measurements, a 400$\times$ improvement in sample efficiency compared to a recent meta-learning approach. Our method also improves the total sample efficiency of accuracy predictors by over an order of magnitude. Finally, we demonstrate the effectiveness of our method for multi-search-space and multi-task accuracy prediction on 28 NAS search spaces and tasks.

翻译：许多硬件感知的神经架构搜索（NAS）方法已被开发出来，旨在优化神经网络（NN）的拓扑结构，同时实现更高精度和更低延迟的联合目标。近年来，精度预测器和延迟预测器在NAS中取得了巨大成功，分别实现了高样本效率和硬件（HW）设备延迟的精确建模。然而，对于每个新的NAS搜索空间或NN任务，都需要重新训练一个精度预测器；对于每个新的硬件设备，则需额外训练一个延迟预测器。本文探索了实现精度和延迟预测器在多任务、多搜索空间及多硬件上自适应的方法，以降低NAS的成本。我们提出了一种基于零成本代理的新型搜索空间无关神经网络编码，该编码能在多个任务和NAS搜索空间上实现样本高效的预测，在多种场景下将延迟和精度预测器的端到端样本效率提升超过一个数量级。例如，我们的神经网络编码能在不到85次硬件测量内实现从NASBench-201到FBNet（反之亦然）的延迟预测器跨搜索空间迁移，与最近的元学习方法相比，样本效率提升了400倍。我们的方法还将精度预测器的总样本效率提高了超过一个数量级。最后，我们在28个NAS搜索空间和任务上验证了该方法在多搜索空间和多任务精度预测中的有效性。