Deep neural architectures have profound impact on achieved performance in many of today's AI tasks, yet, their design still heavily relies on human prior knowledge and experience. Neural architecture search (NAS) together with hyperparameter optimization (HO) helps to reduce this dependence. However, state of the art NAS and HO rapidly become infeasible with increasing amount of data being stored in a distributed fashion, typically violating data privacy regulations such as GDPR and CCPA. As a remedy, we introduce FEATHERS - $\textbf{FE}$derated $\textbf{A}$rchi$\textbf{T}$ecture and $\textbf{H}$yp$\textbf{ER}$parameter $\textbf{S}$earch, a method that not only optimizes both neural architectures and optimization-related hyperparameters jointly in distributed data settings, but further adheres to data privacy through the use of differential privacy (DP). We show that FEATHERS efficiently optimizes architectural and optimization-related hyperparameters alike, while demonstrating convergence on classification tasks at no detriment to model performance when complying with privacy constraints.

翻译：深度神经网络架构对当今众多人工智能任务的性能具有深远影响，然而其设计仍然严重依赖人类先验知识和经验。神经架构搜索（NAS）与超参数优化（HO）有助于减少这种依赖性。然而，当数据以分布式方式存储且规模不断增大时，现有先进的NAS和HO方法往往变得不可行，且通常违反GDPR和CCPA等数据隐私法规。为解决这一问题，我们引入了FEATHERS（$\textbf{FE}$derated $\textbf{A}$rchi$\textbf{T}$ecture and $\textbf{H}$yp$\textbf{ER}$parameter $\textbf{S}$earch，即联邦架构与超参数搜索），该方法不仅能在分布式数据设置中联合优化神经架构和优化相关超参数，还通过差分隐私（DP）技术确保数据隐私。我们证明，FEATHERS能高效优化架构和优化相关超参数，同时在满足隐私约束的条件下，在分类任务上展现出收敛性，且不影响模型性能。