Blending multiple convolutional kernels is proved advantageous in neural architecture design. However, current two-stage neural architecture search methods are mainly limited to single-path search spaces. How to efficiently search models of multi-path structures remains a difficult problem. In this paper, we are motivated to train a one-shot multi-path supernet to accurately evaluate the candidate architectures. Specifically, we discover that in the studied search spaces, feature vectors summed from multiple paths are nearly multiples of those from a single path. Such disparity perturbs the supernet training and its ranking ability. Therefore, we propose a novel mechanism called Shadow Batch Normalization (SBN) to regularize the disparate feature statistics. Extensive experiments prove that SBNs are capable of stabilizing the optimization and improving ranking performance. We call our unified multi-path one-shot approach as MixPath, which generates a series of models that achieve state-of-the-art results on ImageNet.

翻译：混合多个卷积核被证明在神经架构设计中具有优势。然而，当前的两阶段神经架构搜索方法主要局限于单路径搜索空间，如何高效搜索多路径结构模型仍是一个难题。本文旨在训练一个一次性多路径超网，以准确评估候选架构。具体而言，我们发现，在所研究的搜索空间中，多路径求和得到的特征向量与单路径特征向量近似呈倍数关系。这种差异会干扰超网训练及其排序能力。因此，我们提出一种称为影子批归一化（Shadow Batch Normalization, SBN）的新机制，用以规范这种差异化的特征统计。大量实验证明，SBN能够稳定优化过程并提升排序性能。我们将这种统一的多路径一次性方法命名为MixPath，其在ImageNet上生成了一系列达到最先进结果的模型。