The integration of fully homomorphic encryption (FHE) in federated learning (FL) has led to significant advances in data privacy. However, during the aggregation phase, it often results in performance degradation of the aggregated model, hindering the development of robust representational generalization. In this work, we propose a novel multimodal quantum federated learning framework that utilizes quantum computing to counteract the performance drop resulting from FHE. For the first time in FL, our framework combines a multimodal quantum mixture of experts (MQMoE) model with FHE, incorporating multimodal datasets for enriched representation and task-specific learning. Our MQMoE framework enhances performance on multimodal datasets and combined genomics and brain MRI scans, especially for underrepresented categories. Our results also demonstrate that the quantum-enhanced approach mitigates the performance degradation associated with FHE and improves classification accuracy across diverse datasets, validating the potential of quantum interventions in enhancing privacy in FL.

翻译：全同态加密在联邦学习中的集成显著推动了数据隐私保护的发展。然而，在聚合阶段，该技术常导致聚合模型性能下降，阻碍了鲁棒表征泛化能力的发展。本研究提出了一种新颖的多模态量子联邦学习框架，利用量子计算来抵消由全同态加密引起的性能下降。我们的框架首次在联邦学习中结合了多模态量子专家混合模型与全同态加密，并引入多模态数据集以增强表征能力和任务特异性学习。该MQMoE框架在多模态数据集及基因组学与脑部MRI扫描的联合数据上表现出性能提升，尤其对于代表性不足的类别。实验结果进一步表明，量子增强方法有效缓解了全同态加密相关的性能衰减，并在多种数据集上提高了分类准确率，验证了量子技术在增强联邦学习隐私保护方面的潜力。