Federated learning (FL) is a distributed machine learning (ML) paradigm, allowing multiple clients to collaboratively train shared machine learning (ML) models without exposing clients' data privacy. It has gained substantial popularity in recent years, especially since the enforcement of data protection laws and regulations in many countries. To foster the application of FL, a variety of FL frameworks have been proposed, allowing non-experts to easily train ML models. As a result, understanding bugs in FL frameworks is critical for facilitating the development of better FL frameworks and potentially encouraging the development of bug detection, localization and repair tools. Thus, we conduct the first empirical study to comprehensively collect, taxonomize, and characterize bugs in FL frameworks. Specifically, we manually collect and classify 1,119 bugs from all the 676 closed issues and 514 merged pull requests in 17 popular and representative open-source FL frameworks on GitHub. We propose a classification of those bugs into 12 bug symptoms, 12 root causes, and 18 fix patterns. We also study their correlations and distributions on 23 functionalities. We identify nine major findings from our study, discuss their implications and future research directions based on our findings.
翻译:联邦学习(FL)是一种分布式机器学习(ML)范式,允许多个客户端在不暴露数据隐私的前提下协作训练共享的机器学习模型。近年来,随着多国数据保护法规的推行,联邦学习获得了广泛关注。为促进FL的应用,研究人员提出了多种FL框架,使非专业人员也能轻松训练机器学习模型。因此,理解FL框架中的漏洞对于开发更优质的FL框架、推动漏洞检测、定位与修复工具的研发至关重要。本研究首次通过实证方法系统性地收集、分类并刻画了FL框架中的漏洞。具体而言,我们从GitHub上17个代表性开源FL框架的676个已关闭议题和514个合并拉取请求中手动收集并归类了1,119个漏洞。我们提出将漏洞划分为12种症状、12种根本原因和18种修复模式,并分析了它们与23项功能之间的关联分布。基于研究结果,我们总结了9项重要发现,并探讨了其启示意义及未来研究方向。