Performance prediction has been a key part of the neural architecture search (NAS) process, allowing to speed up NAS algorithms by avoiding resource-consuming network training. Although many performance predictors correlate well with ground truth performance, they require training data in the form of trained networks. Recently, zero-cost proxies have been proposed as an efficient method to estimate network performance without any training. However, they are still poorly understood, exhibit biases with network properties, and their performance is limited. Inspired by the drawbacks of zero-cost proxies, we propose neural graph features (GRAF), simple to compute properties of architectural graphs. GRAF offers fast and interpretable performance prediction while outperforming zero-cost proxies and other common encodings. In combination with other zero-cost proxies, GRAF outperforms most existing performance predictors at a fraction of the cost.
翻译:性能预测一直是神经架构搜索(NAS)过程的关键组成部分,通过避免资源消耗型的网络训练来加速NAS算法。尽管许多性能预测器与真实性能具有良好的相关性,但它们需要以已训练网络的形式提供训练数据。最近,零成本代理被提出作为一种无需任何训练即可估计网络性能的高效方法。然而,人们对它们仍知之甚少,它们存在与网络属性相关的偏差,且其性能有限。受零成本代理缺点的启发,我们提出了神经图特征(GRAF),即架构图的简单可计算属性。GRAF在超越零成本代理和其他常见编码方法的同时,提供了快速且可解释的性能预测。与其他零成本代理结合使用时,GRAF以极低的成本超越了大多数现有性能预测器的表现。