Federated learning has become a popular machine learning paradigm with many potential real-life applications, including recommendation systems, the Internet of Things (IoT), healthcare, and self-driving cars. Though most current applications focus on classification-based tasks, learning personalized generative models remains largely unexplored, and their benefits in the heterogeneous setting still need to be better understood. This work proposes a novel architecture combining global client-agnostic and local client-specific generative models. We show that using standard techniques for training federated models, our proposed model achieves privacy and personalization by implicitly disentangling the globally consistent representation (i.e. content) from the client-dependent variations (i.e. style). Using such decomposition, personalized models can generate locally unseen labels while preserving the given style of the client and can predict the labels for all clients with high accuracy by training a simple linear classifier on the global content features. Furthermore, disentanglement enables other essential applications, such as data anonymization, by sharing only the content. Extensive experimental evaluation corroborates our findings, and we also discuss a theoretical motivation for the proposed approach.

翻译：联邦学习已成为一种流行的机器学习范式，在推荐系统、物联网、医疗保健和自动驾驶汽车等众多潜在现实应用中展现出广阔前景。尽管当前大多数应用集中于基于分类的任务，但个性化生成模型的学习在很大程度上仍未得到充分探索，且其在异构环境中的优势仍需深入理解。本研究提出了一种新颖的架构，该架构结合了全局客户端无关生成模型与局部客户端特定生成模型。我们证明，通过采用训练联邦模型的标准技术，所提出的模型能够通过隐式解耦全局一致表示（即内容）与客户端相关变化（即风格），实现隐私保护与个性化。利用这种分解机制，个性化模型能够在保持客户端给定风格的同时生成局部未见过的标签，并且通过在全局内容特征上训练简单的线性分类器，可以高精度预测所有客户端的标签。此外，解耦技术还支持其他关键应用，例如仅通过共享内容实现数据匿名化。广泛的实验评估证实了我们的发现，本文亦对提出方法的理论动机进行了探讨。