The integration of fully homomorphic encryption (FHE) in federated learning (FL) has led to significant advances in data privacy. However, during the aggregation phase, it often results in performance degradation of the aggregated model, hindering the development of robust representational generalization. In this work, we propose a novel multimodal quantum federated learning framework that utilizes quantum computing to counteract the performance drop resulting from FHE. For the first time in FL, our framework combines a multimodal quantum mixture of experts (MQMoE) model with FHE, incorporating multimodal datasets for enriched representation and task-specific learning. Our MQMoE framework enhances performance on multimodal datasets and combined genomics and brain MRI scans, especially for underrepresented categories. Our results also demonstrate that the quantum-enhanced approach mitigates the performance degradation associated with FHE and improves classification accuracy across diverse datasets, validating the potential of quantum interventions in enhancing privacy in FL.

翻译：全同态加密（FHE）在联邦学习（FL）中的整合为数据隐私带来了显著进展。然而，在聚合阶段，它往往导致聚合模型性能下降，阻碍了鲁棒表征泛化能力的发展。本研究提出了一种新颖的多模态量子联邦学习框架，利用量子计算来抵消由FHE引起的性能下降。我们的框架首次在FL中将多模态量子专家混合（MQMoE）模型与FHE相结合，整合多模态数据集以丰富表征并实现任务特定学习。我们的MQMoE框架在多模态数据集以及基因组学与脑部MRI扫描的组合数据上提升了性能，尤其对于代表性不足的类别。实验结果进一步表明，量子增强方法减轻了与FHE相关的性能下降，并在多种数据集上提高了分类准确率，验证了量子干预在增强FL隐私保护方面的潜力。