The integration of fully homomorphic encryption (FHE) in federated learning (FL) has led to significant advances in data privacy. However, during the aggregation phase, it often results in performance degradation of the aggregated model, hindering the development of robust representational generalization. In this work, we propose a novel multimodal quantum federated learning framework that utilizes quantum computing to counteract the performance drop resulting from FHE. For the first time in FL, our framework combines a multimodal quantum mixture of experts (MQMoE) model with FHE, incorporating multimodal datasets for enriched representation and task-specific learning. Our MQMoE framework enhances performance on multimodal datasets and combined genomics and brain MRI scans, especially for underrepresented categories. Our results also demonstrate that the quantum-enhanced approach mitigates the performance degradation associated with FHE and improves classification accuracy across diverse datasets, validating the potential of quantum interventions in enhancing privacy in FL.

翻译：全同态加密（FHE）与联邦学习（FL）的结合在数据隐私保护方面取得了显著进展。然而，在聚合阶段，该方案常导致聚合模型性能下降，阻碍了鲁棒表征泛化能力的发展。本文提出一种新颖的多模态量子联邦学习框架，利用量子计算来抵消由FHE引入的性能损失。本框架首次在联邦学习中，将多模态量子专家混合（MQMoE）模型与FHE相结合，并引入多模态数据集以丰富表征并支持任务特异性学习。我们的MQMoE框架在多模态数据集以及基因组学与脑部MRI扫描的联合数据上均表现出性能提升，尤其对于代表性不足的类别效果显著。实验结果进一步表明，量子增强方法有效缓解了FHE带来的性能下降，并在多种数据集上提升了分类准确率，验证了量子技术干预在增强联邦学习隐私保护方面的潜力。