Predictor-based methods have substantially enhanced Neural Architecture Search (NAS) optimization. The efficacy of these predictors is largely influenced by the method of encoding neural network architectures. While traditional encodings used an adjacency matrix describing the graph structure of a neural network, novel encodings embrace a variety of approaches from unsupervised pretraining of latent representations to vectors of zero-cost proxies. In this paper, we categorize and investigate neural encodings from three main types: structural, learned, and score-based. Furthermore, we extend these encodings and introduce \textit{unified encodings}, that extend NAS predictors to multiple search spaces. Our analysis draws from experiments conducted on over 1.5 million neural network architectures on NAS spaces such as NASBench-101 (NB101), NB201, NB301, Network Design Spaces (NDS), and TransNASBench-101. Building on our study, we present our predictor \textbf{FLAN}: \textbf{Fl}ow \textbf{A}ttention for \textbf{N}AS. FLAN integrates critical insights on predictor design, transfer learning, and \textit{unified encodings} to enable more than an order of magnitude cost reduction for training NAS accuracy predictors. Our implementation and encodings for all neural networks are open-sourced at \href{https://github.com/abdelfattah-lab/flan_nas}{https://github.com/abdelfattah-lab/flan\_nas}.

翻译：预测器方法显著增强了神经架构搜索（NAS）的优化效果。这些预测器的效能很大程度上受神经网络架构编码方法的影响。传统编码使用描述神经网络图结构的邻接矩阵，而新型编码则采用了多种方法，从潜在表示的无监督预训练到零代价代理向量。本文从三个主要类型对神经编码进行分类和研究：结构编码、学习编码和基于分数的编码。此外，我们扩展了这些编码，提出了**统一编码**，使NAS预测器能够扩展到多个搜索空间。我们的分析基于在NASBench-101（NB101）、NB201、NB301、网络设计空间（NDS）和TransNASBench-101等NAS空间上对超过150万个神经网络架构进行的实验。基于我们的研究，我们提出了预测器**FLAN**：用于NAS的**流**注意力（**Fl**ow **A**ttention for **N**AS）。FLAN整合了关于预测器设计、迁移学习和**统一编码**的关键见解，使得训练NAS精度预测器的成本降低了一个数量级以上。我们所有神经网络的实现和编码已在 \href{https://github.com/abdelfattah-lab/flan_nas}{https://github.com/abdelfattah-lab/flan\_nas} 上开源。