Current neural architecture search (NAS) strategies focus only on finding a single, good, architecture. They offer little insight into why a specific network is performing well, or how we should modify the architecture if we want further improvements. We propose a Bayesian optimisation (BO) approach for NAS that combines the Weisfeiler-Lehman graph kernel with a Gaussian process surrogate. Our method optimises the architecture in a highly data-efficient manner: it is capable of capturing the topological structures of the architectures and is scalable to large graphs, thus making the high-dimensional and graph-like search spaces amenable to BO. More importantly, our method affords interpretability by discovering useful network features and their corresponding impact on the network performance. Indeed, we demonstrate empirically that our surrogate model is capable of identifying useful motifs which can guide the generation of new architectures. We finally show that our method outperforms existing NAS approaches to achieve the state of the art on both closed- and open-domain search spaces.

翻译：当前的神经结构搜索(NAS)战略仅侧重于寻找单一的、好的建筑。 它们很少深入了解为什么特定网络运行良好, 或者如果我们想要进一步改进, 我们应该如何修改结构。 我们为NAS提出了一种贝叶斯优化(BO) 方法, 将 Weisfeiler- Lehman 图形内核与 Gausian 进程替代器结合起来。 我们的方法以高数据效率高的方式优化了建筑结构: 它能够捕捉建筑的表层结构, 并且可以向大图表伸缩, 从而使高维和像图形一样的搜索空间适合BO。 更重要的是, 我们的方法通过发现有用的网络特征及其对网络性能的相应影响来提供解释性。 事实上, 我们从经验上表明, 我们的代孕模型能够找到有用的模型来指导新建筑的生成。 我们最后显示, 我们的方法超越了现有NAS 方法, 从而在封闭和开放搜索空间上实现艺术状态。