Conventional centralised deep learning paradigms are not feasible when data from different sources cannot be shared due to data privacy or transmission limitation. To resolve this problem, federated learning has been introduced to transfer knowledge across multiple sources (clients) with non-shared data while optimising a globally generalised central model (server). Existing federated learning paradigms mostly focus on transferring holistic high-level knowledge (such as class) across models, which are closely related to specific objects of interest so may suffer from inverse attack. In contrast, in this work, we consider transferring mid-level semantic knowledge (such as attribute) which is not sensitive to specific objects of interest and therefore is more privacy-preserving and scalable. To this end, we formulate a new Federated Zero-Shot Learning (FZSL) paradigm to learn mid-level semantic knowledge at multiple local clients with non-shared local data and cumulatively aggregate a globally generalised central model for deployment. To improve model discriminative ability, we propose to explore semantic knowledge augmentation from external knowledge for enriching the mid-level semantic space in FZSL. Extensive experiments on five zeroshot learning benchmark datasets validate the effectiveness of our approach for optimising a generalisable federated learning model with mid-level semantic knowledge transfer.

翻译：传统集中式深度学习范式在数据因隐私或传输限制无法跨源共享时不可行。为解决此问题，联邦学习被引入，通过非共享数据在多个源（客户端）间传递知识，同时优化全局泛化的中央模型（服务器）。现有联邦学习范式主要关注跨模型传递整体高层知识（如类别），这类知识与特定目标对象紧密相关，易遭受逆向攻击。相比之下，本研究考虑传递中层级语义知识（如属性），此类知识对特定目标对象不敏感，因此更具隐私保护性和可扩展性。为此，我们提出了一种新的联邦零样本学习（FZSL）范式，用于在多个本地客户端上通过非共享本地数据学习中层级语义知识，并逐步聚合全局泛化的中央模型以供部署。为提升模型判别能力，我们提出从外部知识中探索语义知识增强，以丰富FZSL中的中层级语义空间。在五个零样本学习基准数据集上的大量实验验证了本方法在通过中层级语义知识传递优化泛化联邦学习模型方面的有效性。