The use of Neural Architecture Search (NAS) techniques to automate the design of neural networks has become increasingly popular in recent years. The proliferation of devices with different hardware characteristics using such neural networks, as well as the need to reduce the power consumption for their search, has led to the realisation of Once-For-All (OFA), an eco-friendly algorithm characterised by the ability to generate easily adaptable models through a single learning process. In order to improve this paradigm and develop high-performance yet eco-friendly NAS techniques, this paper presents OFAv2, the extension of OFA aimed at improving its performance while maintaining the same ecological advantage. The algorithm is improved from an architectural point of view by including early exits, parallel blocks and dense skip connections. The training process is extended by two new phases called Elastic Level and Elastic Height. A new Knowledge Distillation technique is presented to handle multi-output networks, and finally a new strategy for dynamic teacher network selection is proposed. These modifications allow OFAv2 to improve its accuracy performance on the Tiny ImageNet dataset by up to 12.07% compared to the original version of OFA, while maintaining the algorithm flexibility and advantages.

翻译：近年来，利用神经架构搜索（NAS）技术实现神经网络自动化设计日益流行。由于不同硬件特性的设备广泛采用此类神经网络，且需降低其搜索功耗，催生了Once-For-All（OFA）这一生态友好型算法——其核心优势在于可通过单次学习过程生成易于适配的模型。为改进该范式并开发高性能且生态友好的NAS技术，本文提出OFAv2——即OFA的扩展版本，旨在保持相同生态优势的同时提升性能。算法从架构层面进行改进，通过引入早期退出机制、并行块与密集跳跃连接实现优化。训练过程新增两个阶段：弹性层级（Elastic Level）与弹性高度（Elastic Height）。针对多输出网络提出新型知识蒸馏技术，并最终设计了动态教师网络选择策略。这些改进使得OFAv2在Tiny ImageNet数据集上相较原始OFA版本最高提升12.07%的准确率性能，同时保持算法灵活性与原有优势。