Deep neural architectures have profound impact on achieved performance in many of today's AI tasks, yet, their design still heavily relies on human prior knowledge and experience. Neural architecture search (NAS) together with hyperparameter optimization (HO) helps to reduce this dependence. However, state of the art NAS and HO rapidly become infeasible with increasing amount of data being stored in a distributed fashion, typically violating data privacy regulations such as GDPR and CCPA. As a remedy, we introduce FEATHERS - $\textbf{FE}$derated $\textbf{A}$rchi$\textbf{T}$ecture and $\textbf{H}$yp$\textbf{ER}$parameter $\textbf{S}$earch, a method that not only optimizes both neural architectures and optimization-related hyperparameters jointly in distributed data settings, but further adheres to data privacy through the use of differential privacy (DP). We show that FEATHERS efficiently optimizes architectural and optimization-related hyperparameters alike, while demonstrating convergence on classification tasks at no detriment to model performance when complying with privacy constraints.

翻译：深度神经架构对当今许多人工智能任务的性能有着深远影响，但其设计仍严重依赖人类先验知识和经验。神经架构搜索（NAS）与超参数优化（HO）有助于减少这种依赖性。然而，随着分布式存储数据量的增加，最先进的NAS和HO方法迅速变得不可行，且通常违反GDPR和CCPA等数据隐私法规。为此，我们提出FEATHERS——联邦架构与超参数搜索，这一方法不仅能在分布式数据环境中联合优化神经架构和与优化相关的超参数，还能通过使用差分隐私（DP）来遵守数据隐私要求。我们表明，FEATHERS能够高效优化架构和与优化相关的超参数，同时在遵守隐私约束的情况下，在分类任务上展现出收敛性，且不损害模型性能。