Federated Learning (FL) is an emerging domain in the broader context of artificial intelligence research. Methodologies pertaining to FL assume distributed model training, consisting of a collection of clients and a server, with the main goal of achieving optimal global model with restrictions on data sharing due to privacy concerns. It is worth highlighting that the diverse existing literature in FL mostly assume stationary data generation processes; such an assumption is unrealistic in real-world conditions where concept drift occurs due to, for instance, seasonal or period observations, faults in sensor measurements. In this paper, we introduce a multiscale algorithmic framework which combines theoretical guarantees of \textit{FedAvg} and \textit{FedOMD} algorithms in near stationary settings with a non-stationary detection and adaptation technique to ameliorate FL generalization performance in the presence of concept drifts. We present a multi-scale algorithmic framework leading to $\Tilde{\mathcal{O}} ( \min \{ \sqrt{LT} , \Delta^{\frac{1}{3}}T^{\frac{2}{3}} + \sqrt{T} \})$ \textit{dynamic regret} for $T$ rounds with an underlying general convex loss function, where $L$ is the number of times non-stationary drifts occurred and $\Delta$ is the cumulative magnitude of drift experienced within $T$ rounds.
翻译:联邦学习是人工智能研究广泛背景下的新兴领域。联邦学习方法涉及分布式模型训练,由一组客户端和服务器组成,其主要目标是在数据共享受隐私限制的条件下实现最优全局模型。值得注意的是,现有联邦学习文献大多假设数据生成过程具有平稳性;然而这一假设在现实条件中并不成立,因为概念漂移可能由季节性/周期性观测或传感器测量故障等因素引发。本文提出一种多尺度算法框架,该框架将近平稳环境下\textit{FedAvg}与\textit{FedOMD}算法的理论保证与非平稳检测及自适应技术相结合,以提升联邦学习在概念漂移情境下的泛化性能。我们提出的多尺度算法框架在底层一般凸损失函数下,实现了$T$轮迭代中$\Tilde{\mathcal{O}} ( \min \{ \sqrt{LT} , \Delta^{\frac{1}{3}}T^{\frac{2}{3}} + \sqrt{T} \})$的\textit{动态遗憾},其中$L$为非平稳漂移发生次数,$\Delta$为$T$轮内累积漂移幅度。