Vertical federated learning has garnered significant attention as it allows clients to train machine learning models collaboratively without sharing local data, which protects the client's local private data. However, existing VFL methods face challenges when dealing with heterogeneous local models among participants, which affects optimization convergence and generalization. To address this challenge, this paper proposes a novel approach called Vertical federated learning for training multiple Heterogeneous models (VFedMH). VFedMH focuses on aggregating the local embeddings of each participant's knowledge during forward propagation. To protect the participants' local embedding values, we propose an embedding protection method based on lightweight blinding factors. In particular, participants obtain local embedding using local heterogeneous models. Then the passive party, who owns only features of the sample, injects the blinding factor into the local embedding and sends it to the active party. The active party aggregates local embeddings to obtain global knowledge embeddings and sends them to passive parties. The passive parties then utilize the global embeddings to propagate forward on their local heterogeneous networks. However, the passive party does not own the sample labels, so the local model gradient cannot be calculated locally. To overcome this limitation, the active party assists the passive party in computing its local heterogeneous model gradients. Then, each participant trains their local model using the heterogeneous model gradients. The objective is to minimize the loss value of their respective local heterogeneous models. Extensive experiments are conducted to demonstrate that VFedMH can simultaneously train multiple heterogeneous models with heterogeneous optimization and outperform some recent methods in model performance.

翻译：纵向联邦学习（VFL）允许客户端在不共享本地数据的情况下协同训练机器学习模型，有效保护客户端的本地私有数据，因而受到广泛关注。然而，现有VFL方法在处理参与者之间的异构本地模型时面临挑战，这会影响优化收敛和泛化性能。为解决该问题，本文提出一种名为VFedMH（面向多异构模型训练的纵向联邦学习）的新方法。VFedMH的核心思想是在前向传播过程中聚合各参与者本地嵌入所蕴含的知识。为保护参与者的本地嵌入值，我们提出一种基于轻量级盲化因子的嵌入保护方法。具体而言，参与者利用本地异构模型获取本地嵌入，随后仅拥有样本特征的被动方将盲化因子注入本地嵌入并发送至主动方。主动方聚合本地嵌入以获取全局知识嵌入，并将其分发给各被动方。被动方利用全局嵌入在其本地异构网络上进行前向传播。然而，由于被动方不拥有样本标签，无法在本地计算模型梯度。为克服这一局限，主动方协助被动方计算其本地异构模型的梯度。此后，每个参与者利用异构模型梯度训练其本地模型，目标是最小化各自本地异构模型的损失值。大量实验表明，VFedMH能够通过异构优化同时训练多个异构模型，并在模型性能上优于近期多种方法。