Neural architecture search has proven to be a powerful approach to designing and refining neural networks, often boosting their performance and efficiency over manually-designed variations, but comes with computational overhead. While there has been a considerable amount of research focused on lowering the cost of NAS for mainstream tasks, such as image classification, a lot of those improvements stem from the fact that those tasks are well-studied in the broader context. Consequently, applicability of NAS to emerging and under-represented domains is still associated with a relatively high cost and/or uncertainty about the achievable gains. To address this issue, we turn our focus towards the recent growth of publicly available NAS benchmarks in an attempt to extract general NAS knowledge, transferable across different tasks and search spaces. We borrow from the rich field of meta-learning for few-shot adaptation and carefully study applicability of those methods to NAS, with a special focus on the relationship between task-level correlation (domain shift) and predictor transferability; which we deem critical for improving NAS on diverse tasks. In our experiments, we use 6 NAS benchmarks in conjunction, spanning in total 16 NAS settings -- our meta-learning approach not only shows superior (or matching) performance in the cross-validation experiments but also successful extrapolation to a new search space and tasks.

翻译：神经架构搜索已被证明是设计和优化神经网络的有效方法，通常能在手动设计的变体基础上提升性能与效率，但会带来计算开销。尽管已有大量研究聚焦于降低主流任务（如图像分类）中NAS的成本，但这些改进很大程度上源于此类任务在更广泛背景下已被充分研究。因此，NAS在新兴及代表性不足领域的应用仍伴随着相对较高的成本和/或对可达收益的不确定性。为解决此问题，我们将目光转向近年来可公开获取的NAS基准测试集，试图提取可跨不同任务和搜索空间迁移的通用NAS知识。我们借鉴了元学习领域中关于少样本适应的丰富研究成果，系统研究了这些方法在NAS中的适用性，特别关注任务级相关性（域偏移）与预测器可迁移性之间的关系——我们认为这对改善多样化任务上的NAS性能至关重要。在实验中，我们联合使用6个NAS基准测试集，涵盖总计16种NAS设置——我们的元学习方法不仅在交叉验证实验中展现出优越（或相当）的性能，还能成功外推至新的搜索空间和任务。