Federated learning enables multiple hospitals to cooperatively learn a shared model without privacy disclosure. Existing methods often take a common assumption that the data from different hospitals have the same modalities. However, such a setting is difficult to fully satisfy in practical applications, since the imaging guidelines may be different between hospitals, which makes the number of individuals with the same set of modalities limited. To this end, we formulate this practical-yet-challenging cross-modal vertical federated learning task, in which shape data from multiple hospitals have different modalities with a small amount of multi-modality data collected from the same individuals. To tackle such a situation, we develop a novel framework, namely Federated Consistent Regularization constrained Feature Disentanglement (Fed-CRFD), for boosting MRI reconstruction by effectively exploring the overlapping samples (individuals with multi-modalities) and solving the domain shift problem caused by different modalities. Particularly, our Fed-CRFD involves an intra-client feature disentangle scheme to decouple data into modality-invariant and modality-specific features, where the modality-invariant features are leveraged to mitigate the domain shift problem. In addition, a cross-client latent representation consistency constraint is proposed specifically for the overlapping samples to further align the modality-invariant features extracted from different modalities. Hence, our method can fully exploit the multi-source data from hospitals while alleviating the domain shift problem. Extensive experiments on two typical MRI datasets demonstrate that our network clearly outperforms state-of-the-art MRI reconstruction methods. The source code will be publicly released upon the publication of this work.

翻译：联邦学习使多家医院能够在不泄露隐私的情况下合作学习共享模型。现有方法通常假设不同医院的数据具有相同模态。然而，这种设置在实际应用中难以完全满足，因为不同医院的成像指南可能存在差异，导致拥有相同模态集合的个体数量有限。为此，我们提出了一项实用且具有挑战性的跨模态垂直联邦学习任务，其中多家医院的多中心数据具有不同模态，仅少量多模态数据来自相同个体。针对这一情况，我们开发了一种名为"联邦一致性正则化约束特征解耦（Fed-CRFD）"的新框架，通过有效探索重叠样本（多模态个体）并解决不同模态引起的域偏移问题，提升MRI重建性能。具体而言，我们的Fed-CRFD引入客户端内特征解耦机制，将数据分离为模态不变特征和模态特定特征，利用模态不变特征缓解域偏移问题。此外，针对重叠样本提出了跨客户端潜在表示一致性约束，进一步对齐从不同模态提取的模态不变特征。因此，本方法既能充分利用医院的多源数据，又能减轻域偏移问题。在两个典型MRI数据集上的大量实验表明，我们的网络明显优于当前最先进的MRI重建方法。代码将在本文发表后公开发布。