The traditional vertical federated learning schema suffers from two main issues: 1) restricted applicable scope to overlapped samples and 2) high system challenge of real-time federated serving, which limits its application to advertising systems. To this end, we advocate a new learning setting Semi-VFL (Vertical Semi-Federated Learning) to tackle these challenge. Semi-VFL is proposed to achieve a practical industry application fashion for VFL, by learning a federation-aware local model which performs better than single-party models and meanwhile maintain the convenience of local-serving. For this purpose, we propose the carefully designed Joint Privileged Learning framework (JPL) to i) alleviate the absence of the passive party's feature and ii) adapt to the whole sample space. Specifically, we build an inference-efficient single-party student model applicable to the whole sample space and meanwhile maintain the advantage of the federated feature extension. New representation distillation methods are designed to extract cross-party feature correlations for both the overlapped and non-overlapped data. We conducted extensive experiments on real-world advertising datasets. The results show that our method achieves the best performance over baseline methods and validate its superiority in the Semi-VFL setting.

翻译：传统的纵向联邦学习模式存在两个主要问题：1）应用范围仅限于重叠样本；2）实时联邦服务面临较高系统挑战，这限制了其在广告系统中的应用。为此，我们提出了一种新的学习范式——纵向半联邦学习（Semi-VFL）以应对这些挑战。Semi-VFL旨在实现纵向联邦学习的实用工业应用方式，通过学习一个具有联邦感知能力的本地模型，该模型在性能上优于单方模型，同时保持本地服务的便利性。基于此，我们设计了精心构建的联合特权学习框架（JPL），以：i）缓解被动方特征缺失问题；ii）适应全样本空间。具体而言，我们构建了一个推理高效的单方学生模型，该模型适用于全样本空间，同时保持联邦特征扩展的优势。我们设计了新的表示蒸馏方法，以提取重叠数据与非重叠数据中的跨方特征相关性。我们在真实广告数据集上进行了大量实验，结果表明我们的方法在基准方法中取得了最佳性能，并验证了其在Semi-VFL设置下的优越性。