In this paper, we propose BNAS-v2 to further improve the efficiency of NAS, embodying both superiorities of BCNN simultaneously. To mitigate the unfair training issue of BNAS, we employ continuous relaxation strategy to make each edge of cell in BCNN relevant to all candidate operations for over-parameterized BCNN construction. Moreover, the continuous relaxation strategy relaxes the choice of a candidate operation as a softmax over all predefined operations. Consequently, BNAS-v2 employs the gradient-based optimization algorithm to simultaneously update every possible path of over-parameterized BCNN, rather than the single sampled one as BNAS. However, continuous relaxation leads to another issue named performance collapse, in which those weight-free operations are prone to be selected by the search strategy. For this consequent issue, two solutions are given: 1) we propose Confident Learning Rate (CLR) that considers the confidence of gradient for architecture weights update, increasing with the training time of over-parameterized BCNN; 2) we introduce the combination of partial channel connections and edge normalization that also can improve the memory efficiency further. Moreover, we denote differentiable BNAS (i.e. BNAS with continuous relaxation) as BNAS-D, BNAS-D with CLR as BNAS-v2-CLR, and partial-connected BNAS-D as BNAS-v2-PC. Experimental results on CIFAR-10 and ImageNet show that 1) BNAS-v2 delivers state-of-the-art search efficiency on both CIFAR-10 (0.05 GPU days that is 4x faster than BNAS) and ImageNet (0.19 GPU days); and 2) the proposed CLR is effective to alleviate the performance collapse issue in both BNAS-D and vanilla differentiable NAS framework.

翻译：在本文中,我们提议BNAS- v2 来进一步提高NAS的效率,同时体现BNN的两种优势。然而,为了缓解BNAS的不公平培训问题,我们采用持续放松战略,使BNNN的所有细胞边缘都与超分数的BNNN建筑工程的所有候选操作相关。此外,持续放松战略会放松选择候选人操作,将其作为所有预定义操作的软体积。因此,BNAS- v2 使用基于梯度的优化算法,同时更新超分数的BNNNN,而不是作为BNAS的单一样本的每一个可能的路径。然而,持续放松导致另一个名为性能崩溃的问题,即这些无重力的操作很容易被搜索战略所选中。对此,我们给出了两个解决方案:(1) 我们提出不定性学习率(CLRR),认为对建筑权重更新的梯度的信心,随着过分数的BNCNNNNA的训练时间增加;(2) 我们引入部分频道连接和边缘搜索的组合,这也能进一步提高记忆效率。 此外,我们注意到CS-CS的S-C-S-S-S 和部分S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S