Early Exit Neural Networks (EENNs) endow astandard Deep Neural Network (DNN) with Early Exit Classifiers (EECs), to provide predictions at intermediate points of the processing when enough confidence in classification is achieved. This leads to many benefits in terms of effectiveness and efficiency. Currently, the design of EENNs is carried out manually by experts, a complex and time-consuming task that requires accounting for many aspects, including the correct placement, the thresholding, and the computational overhead of the EECs. For this reason, the research is exploring the use of Neural Architecture Search (NAS) to automatize the design of EENNs. Currently, few comprehensive NAS solutions for EENNs have been proposed in the literature, and a fully automated, joint design strategy taking into consideration both the backbone and the EECs remains an open problem. To this end, this work presents Neural Architecture Search for Hardware Constrained Early Exit Neural Networks (NACHOS), the first NAS framework for the design of optimal EENNs satisfying constraints on the accuracy and the number of Multiply and Accumulate (MAC) operations performed by the EENNs at inference time. In particular, this provides the joint design of backbone and EECs to select a set of admissible (i.e., respecting the constraints) Pareto Optimal Solutions in terms of best tradeoff between the accuracy and number of MACs. The results show that the models designed by NACHOS are competitive with the state-of-the-art EENNs. Additionally, this work investigates the effectiveness of two novel regularization terms designed for the optimization of the auxiliary classifiers of the EENN

翻译：早退神经网络（EENN）赋予标准深度神经网络（DNN）早退分类器（EEC），使其在处理中间阶段达到足够分类置信度时提供预测，从而在有效性和效率方面带来诸多优势。当前EENN的设计依赖专家手动完成，这是一项复杂且耗时的任务，需综合考虑诸多因素，包括EEC的正确放置、阈值设定及计算开销。因此，研究者正在探索利用神经架构搜索（NAS）自动设计EENN。目前文献中鲜有面向EENN的全面NAS解决方案，而综合考虑骨干网络与EEC的全自动联合设计策略仍是一个开放性问题。为此，本文提出面向硬件约束的早退神经网络的神经架构搜索（NACHOS）——首个能在推理时满足EENN准确率与乘累加操作（MAC）数量约束的最优EENN设计的NAS框架。具体而言，该框架通过联合设计骨干网络与EEC，选取一组在准确率与MAC数量间取得最佳权衡的可行（即满足约束）帕累托最优解。结果表明，NACHOS设计的模型与当前最先进的EENN具有竞争力。此外，本文还探究了两种专为优化EENN辅助分类器而设计的新正则化项的有效性。