Neural Architecture Search (NAS) for Federated Learning (FL) is an emerging field. It automates the design and training of Deep Neural Networks (DNNs) when data cannot be centralized due to privacy, communication costs, or regulatory restrictions. Recent federated NAS methods not only reduce manual effort but also help achieve higher accuracy than traditional FL methods like FedAvg. Despite the success, existing federated NAS methods still fall short in satisfying diverse deployment targets common in on-device inference like hardware, latency budgets, or variable battery levels. Most federated NAS methods search for only a limited range of neuro-architectural patterns, repeat them in a DNN, thereby restricting achievable performance. Moreover, these methods incur prohibitive training costs to satisfy deployment targets. They perform the training and search of DNN architectures repeatedly for each case. SuperFedNAS addresses these challenges by decoupling the training and search in federated NAS. SuperFedNAS co-trains a large number of diverse DNN architectures contained inside one supernet in the FL setting. Post-training, clients perform NAS locally to find specialized DNNs by extracting different parts of the trained supernet with no additional training. SuperFedNAS takes O(1) (instead of O(N)) cost to find specialized DNN architectures in FL for any N deployment targets. As part of SuperFedNAS, we introduce MaxNet - a novel FL training algorithm that performs multi-objective federated optimization of a large number of DNN architectures ($\approx 5*10^8$) under different client data distributions. Overall, SuperFedNAS achieves upto 37.7% higher accuracy for the same MACs or upto 8.13x reduction in MACs for the same accuracy than existing federated NAS methods.

翻译：联邦学习（FL）中的神经架构搜索（NAS）是一个新兴领域。当数据因隐私、通信成本或监管限制而无法集中时，它能够自动化深度神经网络（DNN）的设计与训练。近期的联邦NAS方法不仅减少了人工工作量，而且相较于FedAvg等传统联邦学习方法，有助于实现更高的准确率。尽管取得了成功，现有的联邦NAS方法在满足设备端推理中常见的多样化部署目标（如硬件、延迟预算或可变电池电量）方面仍显不足。大多数联邦NAS方法仅搜索有限的神经架构模式，并在DNN中重复使用，从而限制了可实现的性能。此外，这些方法为满足部署目标会产生高昂的训练成本，需要针对每个案例反复进行DNN架构的训练与搜索。SuperFedNAS通过解耦联邦NAS中的训练与搜索来应对这些挑战。该方法在联邦学习设置下，共同训练一个超网中包含的大量多样化DNN架构。训练完成后，客户端通过从已训练的超网中提取不同部分（无需额外训练）在本地执行NAS，以寻找专用DNN。对于任意N个部署目标，SuperFedNAS在联邦学习中寻找专用DNN架构的成本为O(1)（而非O(N)）。作为SuperFedNAS的一部分，我们提出了MaxNet——一种新颖的联邦学习训练算法，可在不同客户端数据分布下对大量DNN架构（约5*10^8）执行多目标联邦优化。总体而言，与现有联邦NAS方法相比，SuperFedNAS在相同MACs下可实现高达37.7%的准确率提升，或在相同准确率下实现高达8.13倍的MACs降低。