Decentralized deep learning requires dealing with non-iid data across clients, which may also change over time due to temporal shifts. While non-iid data has been extensively studied in distributed settings, temporal shifts have received no attention. To the best of our knowledge, we are first with tackling the novel and challenging problem of decentralized learning with non-iid and dynamic data. We propose a novel algorithm that can automatically discover and adapt to the evolving concepts in the network, without any prior knowledge or estimation of the number of concepts. We evaluate our algorithm on standard benchmark datasets and demonstrate that it outperforms previous methods for decentralized learning.

翻译：去中心化深度学习需要处理客户端间非独立同分布的数据，这些数据也可能因时间偏移而随时间变化。尽管非独立同分布在分布式环境中已被广泛研究，但时间偏移尚未得到关注。据我们所知，我们是首个解决非独立同分布与动态数据这一新颖且具有挑战性的去中心化学习问题的工作。我们提出了一种新型算法，该算法无需对概念数量进行先验知识或估计，即可自动发现并适应网络中不断演变的概念。我们在标准基准数据集上评估了该算法，并证明其在去中心化学习中优于以往方法。