Language models trained via federated learning (FL) demonstrate impressive capabilities in handling complex tasks while protecting user privacy. Recent studies indicate that leveraging gradient information and prior knowledge can potentially reveal training samples within FL setting. However, these investigations have overlooked the potential privacy risks tied to the intrinsic architecture of the models. This paper presents a two-stage privacy attack strategy that targets the vulnerabilities in the architecture of contemporary language models, significantly enhancing attack performance by initially recovering certain feature directions as additional supervisory signals. Our comparative experiments demonstrate superior attack performance across various datasets and scenarios, highlighting the privacy leakage risk associated with the increasingly complex architectures of language models. We call for the community to recognize and address these potential privacy risks in designing large language models.

翻译：通过联邦学习（FL）训练的语言模型在处理复杂任务时展现出卓越能力，同时保护用户隐私。近期研究表明，利用梯度信息和先验知识在联邦学习场景中可能揭示训练样本。然而，这些研究忽略了与模型固有架构相关的潜在隐私风险。本文提出一种两阶段隐私攻击策略，针对当代语言模型架构中的脆弱性，通过初步恢复某些特征方向作为额外监督信号，显著提升攻击性能。我们的对比实验在多数据集和多场景下证明了优越的攻击表现，凸显了语言模型日益复杂架构带来的隐私泄露风险。我们呼吁学界在设计大型语言模型时识别并应对这些潜在隐私风险。