Differentiable Neural Architecture Search (NAS) provides efficient, gradient-based methods for automatically designing neural networks, yet its adoption remains limited in practice. We present MIDAS, a novel approach that modernizes DARTS by replacing static architecture parameters with dynamic, input-specific parameters computed via self-attention. To improve robustness, MIDAS (i) localizes the architecture selection by computing it separately for each spatial patch of the activation map, and (ii) introduces a parameter-free, topology-aware search space that models node connectivity and simplifies selecting the two incoming edges per node. We evaluate MIDAS on the DARTS, NAS-Bench-201, and RDARTS search spaces. In DARTS, it reaches 97.42% top-1 on CIFAR-10 and 83.38% on CIFAR-100. In NAS-Bench-201, it consistently finds globally optimal architectures. In RDARTS, it sets the state of the art on two of four search spaces on CIFAR-10. We further analyze why MIDAS works, showing that patchwise attention improves discrimination among candidate operations, and the resulting input-specific parameter distributions are class-aware and predominantly unimodal, providing reliable guidance for decoding.
翻译:可微分神经架构搜索(NAS)提供了基于梯度的有效方法来自动设计神经网络,但其在实际应用中的采用仍然有限。我们提出MIDAS,这是一种新颖的方法,它通过将静态架构参数替换为由自注意力计算的动态、输入特定参数,从而现代化DARTS。为提高鲁棒性,MIDAS(i)通过为激活图的每个空间补丁单独计算架构选择来实现局部化选择;(ii)引入了一种无参数、拓扑感知的搜索空间,该空间建模节点连接性并简化了每个节点的两条入边的选择。我们在DARTS、NAS-Bench-201和RDARTS搜索空间上评估MIDAS。在DARTS中,它在CIFAR-10上达到97.42%的top-1准确率,在CIFAR-100上达到83.38%。在NAS-Bench-201中,它始终能找到全局最优架构。在RDARTS中,它在CIFAR-10的四个搜索空间中的两个上取得了最先进的性能。我们进一步分析了MIDAS有效的原因,表明补丁级注意力提高了候选操作之间的区分度,并且产生的输入特定参数分布具有类别感知性且主要为单峰分布,为解码提供了可靠的指导。