With the increasing number of new neural architecture designs and substantial existing neural architectures, it becomes difficult for the researchers to situate their contributions compared with existing neural architectures or establish the connections between their designs and other relevant ones. To discover similar neural architectures in an efficient and automatic manner, we define a new problem Neural Architecture Retrieval which retrieves a set of existing neural architectures which have similar designs to the query neural architecture. Existing graph pre-training strategies cannot address the computational graph in neural architectures due to the graph size and motifs. To fulfill this potential, we propose to divide the graph into motifs which are used to rebuild the macro graph to tackle these issues, and introduce multi-level contrastive learning to achieve accurate graph representation learning. Extensive evaluations on both human-designed and synthesized neural architectures demonstrate the superiority of our algorithm. Such a dataset which contains 12k real-world network architectures, as well as their embedding, is built for neural architecture retrieval.

翻译：随着新型神经架构设计的不断涌现以及大量已有神经架构的积累，研究者已难以精准定位自身贡献与现有架构的差异，或建立其设计与其他相关架构之间的关联。为高效自动化地发现相似神经架构，我们定义了一个新问题——神经架构检索，即从现有神经架构中检索与查询架构具有相似设计的架构集合。针对神经架构中的计算图，现有图预训练策略因图规模与结构基元限制而无法直接应用。为实现这一目标，我们提出将计算图分割为结构基元，并利用这些基元重建宏观图以解决上述问题，同时引入多级对比学习实现精确的图表示学习。在人工设计与合成神经架构上的广泛评估验证了本算法的优越性。我们构建了一个包含1.2万个真实网络架构及其嵌入表示的数据集，专用于神经架构检索任务。