Language models trained via federated learning (FL) demonstrate impressive capabilities in handling complex tasks while protecting user privacy. Recent studies indicate that leveraging gradient information and prior knowledge can potentially reveal training samples within FL setting. However, these investigations have overlooked the potential privacy risks tied to the intrinsic architecture of the models. This paper presents a two-stage privacy attack strategy that targets the vulnerabilities in the architecture of contemporary language models, significantly enhancing attack performance by initially recovering certain feature directions as additional supervisory signals. Our comparative experiments demonstrate superior attack performance across various datasets and scenarios, highlighting the privacy leakage risk associated with the increasingly complex architectures of language models. We call for the community to recognize and address these potential privacy risks in designing large language models.

翻译：通过联邦学习训练的语言模型在保护用户隐私的同时展现出处理复杂任务的卓越能力。近期研究表明，利用梯度信息和先验知识有可能在联邦学习设置下揭示训练样本。然而，这些研究忽视了与模型内在架构相关的潜在隐私风险。本文提出一种针对当代语言模型架构漏洞的两阶段隐私攻击策略，通过初步恢复特定特征方向作为额外监督信号，显著提升攻击性能。我们的对比实验证明，该方法在多种数据集和场景下均展现出优越的攻击性能，凸显了语言模型日益复杂的架构所伴随的隐私泄露风险。我们呼吁学界在大型语言模型设计中识别并应对这些潜在的隐私风险。