Federated Neural Architecture Search (FedNAS) aims to automate model design for privacy-preserving Federated Learning (FL) but currently faces two critical bottlenecks: unguided supernet training that yields suboptimal models, and costly multi-hour pipelines for post-training subnet discovery. We introduce DeepFedNAS, a novel, two-phase framework underpinned by a multi-objective fitness function that synthesizes mathematical network design with architectural heuristics. Enabled by a re-engineered supernet, DeepFedNAS introduces Federated Pareto Optimal Supernet Training, which leverages a pre-computed Pareto-optimal cache of high-fitness architectures as an intelligent curriculum to optimize shared supernet weights. Subsequently, its Predictor-Free Search Method eliminates the need for costly accuracy surrogates by utilizing this fitness function as a direct, zero-cost proxy for accuracy, enabling on-demand subnet discovery in mere seconds. DeepFedNAS achieves state-of-the-art accuracy (e.g., up to 1.21% absolute improvement on CIFAR-100), superior parameter and communication efficiency, and a substantial ~61x speedup in total post-training search pipeline time. By reducing the pipeline from over 20 hours to approximately 20 minutes (including initial cache generation) and enabling 20-second individual subnet searches, DeepFedNAS makes hardware-aware FL deployments instantaneous and practical. The complete source code and experimental scripts are available at: https://github.com/bostankhan6/DeepFedNAS
翻译:联邦神经架构搜索旨在为保护隐私的联邦学习自动化模型设计,但目前面临两个关键瓶颈:无引导的超网训练导致模型性能欠佳,以及训练后子网发现流程耗时数小时。我们提出了DeepFedNAS,一个新颖的两阶段框架,其核心是一个多目标适应度函数,该函数将数学化网络设计与架构启发式方法相结合。通过重构的超网,DeepFedNAS引入了联邦帕累托最优超网训练,该方法利用预计算的高适应度架构帕累托最优缓存作为智能课程来优化共享超网权重。随后,其无需预测器的搜索方法通过将此适应度函数直接用作零成本的精度代理,消除了对昂贵精度代理模型的需求,从而能在数秒内实现按需子网发现。DeepFedNAS实现了最先进的精度(例如,在CIFAR-100上绝对精度提升高达1.21%)、卓越的参数与通信效率,以及训练后总搜索流程时间约61倍的显著加速。通过将整个流程从超过20小时缩短至约20分钟(包括初始缓存生成),并实现20秒的单个子网搜索,DeepFedNAS使得硬件感知的联邦学习部署即时且实用。完整的源代码和实验脚本可在以下网址获取:https://github.com/bostankhan6/DeepFedNAS