Distributed learning, which does not require gathering training data in a central location, has become increasingly important in the big-data era. In particular, random-walk-based decentralized algorithms are flexible in that they do not need a central server trusted by all clients and do not require all clients to be active in all iterations. However, existing distributed learning algorithms assume that all learning clients share the same task. In this paper, we consider the more difficult meta-learning setting, in which different clients perform different (but related) tasks with limited training data. To reduce communication cost and allow better privacy protection, we propose LoDMeta (Local Decentralized Meta-learning) with the use of local auxiliary optimization parameters and random perturbations on the model parameter. Theoretical results are provided on both convergence and privacy analysis. Empirical results on a number of few-shot learning data sets demonstrate that LoDMeta has similar meta-learning accuracy as centralized meta-learning algorithms, but does not require gathering data from each client and is able to better protect data privacy for each client.

翻译：分布式学习无需将训练数据集中存储，在大数据时代日益重要。特别是基于随机游走的分散式算法具有灵活性，既不需要所有客户端信任的中心服务器，也不要求所有客户端在每次迭代中都处于活跃状态。然而，现有的分布式学习算法假设所有学习客户端执行相同的任务。本文考虑更具挑战性的元学习场景，其中不同客户端使用有限训练数据执行不同（但相关）的任务。为降低通信成本并增强隐私保护，我们提出LoDMeta（本地分散式元学习），该方法利用本地辅助优化参数并对模型参数添加随机扰动。我们在收敛性和隐私分析方面提供了理论结果。在多个少样本学习数据集上的实验结果表明，LoDMeta具有与集中式元学习算法相近的元学习精度，但无需从各客户端收集数据，并能更好地保护每个客户端的数据隐私。