Machine learning practitioners often fine-tune generative pre-trained models like GPT-3 to improve model performance at specific tasks. Previous works, however, suggest that fine-tuned machine learning models memorize and emit sensitive information from the original fine-tuning dataset. Companies such as OpenAI offer fine-tuning services for their models, but no prior work has conducted a memorization attack on any closed-source models. In this work, we simulate a privacy attack on GPT-3 using OpenAI's fine-tuning API. Our objective is to determine if personally identifiable information (PII) can be extracted from this model. We (1) explore the use of naive prompting methods on a GPT-3 fine-tuned classification model, and (2) we design a practical word generation task called Autocomplete to investigate the extent of PII memorization in fine-tuned GPT-3 within a real-world context. Our findings reveal that fine-tuning GPT3 for both tasks led to the model memorizing and disclosing critical personally identifiable information (PII) obtained from the underlying fine-tuning dataset. To encourage further research, we have made our codes and datasets publicly available on GitHub at: https://github.com/albertsun1/gpt3-pii-attacks

翻译：机器学习从业者常常对生成式预训练模型（如GPT-3）进行微调，以提升其在特定任务上的性能。然而，先前的研究表明，微调后的机器学习模型会记忆并从原始微调数据集中泄露敏感信息。诸如OpenAI等公司为其模型提供微调服务，但尚无先例对任何闭源模型实施记忆攻击。本研究利用OpenAI的微调API对GPT-3模拟了一次隐私攻击，旨在探究是否可以从该模型中提取个人身份信息（PII）。我们（1）探索了对GPT-3微调分类模型采用朴素提示方法的效果，以及（2）设计了一项名为Autocomplete的实用词生成任务，以在真实场景中考察微调后GPT-3对PII的记忆程度。研究结果表明，对GPT-3进行两项任务的微调均导致模型记忆并泄露了从基础微调数据集中获取的关键个人身份信息。为鼓励进一步研究，我们已在GitHub上公开了代码和数据集，地址为：https://github.com/albertsun1/gpt3-pii-attacks