Many problems in machine learning rely on multi-task learning (MTL), in which the goal is to solve multiple related machine learning tasks simultaneously. MTL is particularly relevant for privacy-sensitive applications in areas such as healthcare, finance, and IoT computing, where sensitive data from multiple, varied sources are shared for the purpose of learning. In this work, we formalize notions of client-level privacy for MTL via joint differential privacy (JDP), a relaxation of differential privacy for mechanism design and distributed optimization. We then propose an algorithm for mean-regularized MTL, an objective commonly used for applications in personalized federated learning, subject to JDP. We analyze our objective and solver, providing certifiable guarantees on both privacy and utility. Empirically, we find that our method provides improved privacy/utility trade-offs relative to global baselines across common federated learning benchmarks.

翻译：机器学习中的许多问题依赖于多任务学习（MTL），其目标是同时求解多个相关的机器学习任务。MTL在医疗、金融和物联网计算等隐私敏感型应用中尤为重要，这些场景中来自多个不同来源的敏感数据会被共享以进行学习。在本工作中，我们通过联合差分隐私（JDP）——一种针对机制设计和分布式优化的差分隐私松弛变体——形式化了MTL中客户端级别的隐私概念。随后，我们提出了一种适用于均值正则化MTL的算法，该目标函数常用于个性化联邦学习应用，并需满足JDP约束。我们对所提出的目标函数及求解器进行了分析，提供了可验证的隐私与效用保证。实验表明，相较于通用基线方法，我们的方法在常见的联邦学习基准测试中实现了更优的隐私/效用权衡。