Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications. In this paper, we study the privacy implications of fine-tuning LLMs on user data. To this end, we consider a realistic threat model, called user inference, wherein an attacker infers whether or not a user's data was used for fine-tuning. We design attacks for performing user inference that require only black-box access to the fine-tuned LLM and a few samples from a user which need not be from the fine-tuning dataset. We find that LLMs are susceptible to user inference across a variety of fine-tuning datasets, at times with near perfect attack success rates. Further, we theoretically and empirically investigate the properties that make users vulnerable to user inference, finding that outlier users, users with identifiable shared features between examples, and users that contribute a large fraction of the fine-tuning data are most susceptible to attack. Based on these findings, we identify several methods for mitigating user inference including training with example-level differential privacy, removing within-user duplicate examples, and reducing a user's contribution to the training data. While these techniques provide partial mitigation of user inference, we highlight the need to develop methods to fully protect fine-tuned LLMs against this privacy risk.

翻译：微调是为大型语言模型（LLMs）定制专业任务和应用的常见且有效的方法。本文研究了在用户数据上微调LLMs的隐私影响。为此，我们考虑一种称为用户推断的现实威胁模型，其中攻击者推断用户的数据是否被用于微调。我们设计了执行用户推断的攻击方法，该方法仅需对微调后的LLM进行黑盒访问，以及来自用户的少量样本，这些样本不必来自微调数据集。我们发现，在各种微调数据集上，LLMs容易受到用户推断攻击，有时攻击成功率接近完美。此外，我们从理论和实证角度研究了使用户易受用户推断攻击的特性，发现异常用户、示例间具有可识别共享特征的用户以及贡献大量微调数据的用户最易受到攻击。基于这些发现，我们确定了若干缓解用户推断攻击的方法，包括采用示例级差分隐私训练、删除用户内重复示例以及减少用户在训练数据中的贡献。尽管这些技术能部分缓解用户推断攻击，但我们强调仍需开发完全保护微调后LLMs免受此隐私风险的方法。