Federated Learning (FL) is a privacy-preserving distributed machine learning approach geared towards applications in edge devices. However, the problem of designing custom neural architectures in federated environments is not tackled from the perspective of overall system efficiency. In this paper, we propose DC-NAS -- a divide-and-conquer approach that performs supernet-based Neural Architecture Search (NAS) in a federated system by systematically sampling the search space. We propose a novel diversified sampling strategy that balances exploration and exploitation of the search space by initially maximizing the distance between the samples and progressively shrinking this distance as the training progresses. We then perform channel pruning to reduce the training complexity at the devices further. We show that our approach outperforms several sampling strategies including Hadamard sampling, where the samples are maximally separated. We evaluate our method on the CIFAR10, CIFAR100, EMNIST, and TinyImagenet benchmarks and show a comprehensive analysis of different aspects of federated learning such as scalability, and non-IID data. DC-NAS achieves near iso-accuracy as compared to full-scale federated NAS with 50% fewer resources.

翻译：联邦学习（FL）是一种面向边缘设备应用的隐私保护分布式机器学习方法。然而，在联邦环境下设计定制神经架构的问题尚未从整体系统效率的角度得到解决。本文提出DC-NAS——一种采用分治策略的方法，通过在联邦系统中系统性地采样搜索空间来执行基于超网络的神经架构搜索（NAS）。我们提出了一种新颖的多样化采样策略，通过初始最大化样本间距离并随训练进程逐步缩小该距离，来平衡搜索空间的探索与利用。随后执行通道剪枝以进一步降低设备端的训练复杂度。实验表明，我们的方法优于包括哈达玛采样（使样本最大程度分离）在内的多种采样策略。我们在CIFAR10、CIFAR100、EMNIST和TinyImagenet基准数据集上进行了评估，并对联邦学习的可扩展性及非独立同分布数据等不同方面进行了全面分析。与全规模联邦NAS相比，DC-NAS在资源消耗减少50%的情况下实现了接近同等精度的性能。