A central tenet of Federated learning (FL), which trains models without centralizing user data, is privacy. However, previous work has shown that the gradient updates used in FL can leak user information. While the most industrial uses of FL are for text applications (e.g. keystroke prediction), nearly all attacks on FL privacy have focused on simple image classifiers. We propose a novel attack that reveals private user text by deploying malicious parameter vectors, and which succeeds even with mini-batches, multiple users, and long sequences. Unlike previous attacks on FL, the attack exploits characteristics of both the Transformer architecture and the token embedding, separately extracting tokens and positional embeddings to retrieve high-fidelity text. This work suggests that FL on text, which has historically been resistant to privacy attacks, is far more vulnerable than previously thought.

翻译：摘要：联邦学习（FL）的核心原则是隐私保护，因为其在不集中用户数据的情况下训练模型。然而，先前研究表明，FL中使用的梯度更新可能泄露用户信息。尽管FL在工业中最常用于文本应用（例如键盘预测），但几乎所有针对FL隐私的攻击都集中在简单的图像分类器上。我们提出了一种新颖的攻击方法，通过部署恶意参数向量来揭示用户私有文本，该方法即使在包含小批量、多用户和长序列的场景下也能成功。与先前针对FL的攻击不同，该攻击利用了Transformer架构和词嵌入的双重特性，分别提取令牌和位置嵌入以恢复高保真文本。这项工作表明，历史上对隐私攻击具有抵抗力的文本领域FL，实际上远比此前认为的更加脆弱。