Once-for-All (OFA) is a Neural Architecture Search (NAS) framework designed to address the problem of searching efficient architectures for devices with different resources constraints by decoupling the training and the searching stages. The computationally expensive process of training the OFA neural network is done only once, and then it is possible to perform multiple searches for subnetworks extracted from this trained network according to each deployment scenario. In this work we aim to give one step further in the search for efficiency by explicitly conceiving the search stage as a multi-objective optimization problem. A Pareto frontier is then populated with efficient, and already trained, neural architectures exhibiting distinct trade-offs among the conflicting objectives. This could be achieved by using any multi-objective evolutionary algorithm during the search stage, such as NSGA-II and SMS-EMOA. In other words, the neural network is trained once, the searching for subnetworks considering different hardware constraints is also done one single time, and then the user can choose a suitable neural network according to each deployment scenario. The conjugation of OFA and an explicit algorithm for multi-objective optimization opens the possibility of a posteriori decision-making in NAS, after sampling efficient subnetworks which are a very good approximation of the Pareto frontier, given that those subnetworks are already trained and ready to use. The source code and the final search algorithm will be released at https://github.com/ito-rafael/once-for-all-2

翻译：一次性（Once-for-All，OFA）是一种神经架构搜索（NAS）框架，旨在通过解耦训练与搜索阶段，解决在不同资源约束设备上搜索高效架构的问题。OFA神经网络的计算密集型训练过程仅执行一次，随后即可根据各类部署场景，多次搜索从该训练网络中提取的子网络。本研究旨在明确将搜索阶段构想为多目标优化问题，从而在效率提升上更进一步。通过此方法，帕累托前沿将包含具备不同冲突目标间权衡关系的高效且已训练的神经架构。搜索阶段可采用任意多目标进化算法（如NSGA-II和SMS-EMOA）实现目标。换言之，神经网络仅训练一次，针对不同硬件约束的子网络搜索也仅执行一次，而后用户便可根据各部署场景选择合适的神经网络。OFA与显式多目标优化算法的结合，为NAS中的后验决策提供了可能——在采样到帕累托前沿极佳近似的高效子网络后（这些子网络已训练完毕并可随时使用），即可进行决策。源代码及最终搜索算法将在https://github.com/ito-rafael/once-for-all-2 发布。