Federated Learning with Model Distillation (FedMD) is a nascent collaborative learning paradigm, where only output logits of public datasets are transmitted as distilled knowledge, instead of passing on private model parameters that are susceptible to gradient inversion attacks, a known privacy risk in federated learning. In this paper, we found that even though sharing output logits of public datasets is safer than directly sharing gradients, there still exists a substantial risk of data exposure caused by carefully designed malicious attacks. Our study shows that a malicious server can inject a PLI (Paired-Logits Inversion) attack against FedMD and its variants by training an inversion neural network that exploits the confidence gap between the server and client models. Experiments on multiple facial recognition datasets validate that under FedMD-like schemes, by using paired server-client logits of public datasets only, the malicious server is able to reconstruct private images on all tested benchmarks with a high success rate.

翻译：联邦蒸馏学习（FedMD）是一种新兴的协作学习范式，在该范式中仅传输公共数据集的输出逻辑值作为蒸馏知识，而非传递易受梯度反演攻击（联邦学习中已知的隐私风险）影响的私有模型参数。本文发现，尽管共享公共数据集的输出逻辑值比直接共享梯度更安全，但精心设计的恶意攻击仍会导致严重的数据暴露风险。研究表明，恶意服务器可通过训练利用服务器与客户端模型之间置信度差距的反演神经网络，对FedMD及其变体发起PLI（成对逻辑值反演）攻击。在多个面部识别数据集上的实验验证表明：在同类FedMD方案下，仅使用公共数据集的成对服务器-客户端逻辑值，恶意服务器便能在所有测试基准上以高成功率重建私有图像。