Large Language Models (LLMs) have a privacy concern because they memorize training data (including personally identifiable information (PII) like emails and phone numbers) and leak it during inference. A company can train an LLM on its domain-customized data which can potentially also include their users' PII. In order to comply with privacy laws such as the "right to be forgotten", the data points of users that are most vulnerable to extraction could be deleted. We find that once the most vulnerable points are deleted, a new set of points become vulnerable to extraction. So far, little attention has been given to understanding memorization for fine-tuned models. In this work, we also show that not only do fine-tuned models leak their training data but they also leak the pre-training data (and PII) memorized during the pre-training phase. The property of new data points becoming vulnerable to extraction after unlearning and leakage of pre-training data through fine-tuned models can pose significant privacy and legal concerns for companies that use LLMs to offer services. We hope this work will start an interdisciplinary discussion within AI and law communities regarding the need for policies to tackle these issues.

翻译：大型语言模型（LLMs）存在隐私问题，因为它们会记忆训练数据（包括电子邮件和电话号码等个人身份信息），并在推理过程中泄露这些信息。企业可以在其领域定制数据上训练LLM，这些数据可能包含用户的个人身份信息。为遵守"被遗忘权"等隐私法规，最易被提取的用户数据点可被删除。我们发现，一旦最易被提取的数据点被删除，新的数据点又会变得易于提取。目前，针对微调模型记忆机制的研究很少受到关注。本研究还表明，微调模型不仅会泄露其训练数据，还会泄露预训练阶段记忆的预训练数据（及个人身份信息）。遗忘后新数据点变得易被提取的特性，以及通过微调模型泄露预训练数据的现象，可能对使用LLM提供服务的企业构成重大隐私与法律风险。我们希望这项研究能推动人工智能与法律界就制定相关政策应对这些问题展开跨学科讨论。