In the last decade, most research in Machine Learning contributed to the improvement of existing models, with the aim of increasing the performance of neural networks for the solution of a variety of different tasks. However, such advancements often come at the cost of an increase of model memory and computational requirements. This represents a significant limitation for the deployability of research output in realistic settings, where the cost, the energy consumption, and the complexity of the framework play a crucial role. To solve this issue, the designer should search for models that maximise the performance while limiting its footprint. Typical approaches to reach this goal rely either on manual procedures, which cannot guarantee the optimality of the final design, or upon Neural Architecture Search algorithms to automatise the process, at the expenses of extremely high computational time. This paper provides a solution for the fast identification of a neural network that maximises the model accuracy while preserving size and computational constraints typical of tiny devices. Our approach, named FreeREA, is a custom cell-based evolution NAS algorithm that exploits an optimised combination of training-free metrics to rank architectures during the search, thus without need of model training. Our experiments, carried out on the common benchmarks NAS-Bench-101 and NATS-Bench, demonstrate that i) FreeREA is a fast, efficient, and effective search method for models automatic design; ii) it outperforms State of the Art training-based and training-free techniques in all the datasets and benchmarks considered, and iii) it can easily generalise to constrained scenarios, representing a competitive solution for fast Neural Architecture Search in generic constrained applications. The code is available at \url{https://github.com/NiccoloCavagnero/FreeREA}.

翻译：在过去十年中，机器学习领域的大多数研究致力于改进现有模型，以提高神经网络在解决各类任务时的性能。然而，此类进展往往伴随着模型内存和计算需求的增加。这严重制约了研究成果在现实场景中的部署——在这些场景中，成本、能耗和框架复杂度起着关键作用。为解决此问题，设计者需搜索能在最大化性能的同时限制模型规模的架构。此类目标的典型方法要么依赖手动流程（无法保证最终设计的最优性），要么借助神经架构搜索算法实现自动化（但需消耗极高的计算时间）。本文提出一种快速识别神经网络的方案，该网络能在保持微型设备典型尺寸和计算约束的前提下，最大化模型精度。我们的方法命名为FreeREA，是一种基于细胞的自定义进化神经架构搜索算法，它利用免训练指标的优化组合在搜索过程中对架构进行排名，因此无需模型训练。我们在通用基准测试NAS-Bench-101和NATS-Bench上开展的实验表明：（i）FreeREA是一种快速、高效且有效的模型自动设计搜索方法；（ii）它在所有考虑的数据集和基准测试中均优于基于训练和免训练的现有最优技术；（iii）它易于泛化至受约束场景，是通用约束应用中快速神经架构搜索的竞争性解决方案。代码发布于\url{https://github.com/NiccoloCavagnero/FreeREA}。