The latest and most impactful advances in large models stem from their increased size. Unfortunately, this translates into an improved memorization capacity, raising data privacy concerns. Specifically, it has been shown that models can output personal identifiable information (PII) contained in their training data. However, reported PIII extraction performance varies widely, and there is no consensus on the optimal methodology to evaluate this risk, resulting in underestimating realistic adversaries. In this work, we empirically demonstrate that it is possible to improve the extractability of PII by over ten-fold by grounding the prefix of the manually constructed extraction prompt with in-domain data. Our approach, PII-Compass, achieves phone number extraction rates of 0.92%, 3.9%, and 6.86% with 1, 128, and 2308 queries, respectively, i.e., the phone number of 1 person in 15 is extractable.

翻译：大型模型最新且最具影响力的进展源于其规模的增大。不幸的是，这转化为其记忆能力的提升，从而引发了数据隐私方面的担忧。具体而言，已有研究表明模型能够输出其训练数据中包含的个人可识别信息（PII）。然而，已报道的PII提取性能差异巨大，且对于评估此风险的最佳方法尚无共识，导致对现实攻击者的风险被低估。在本工作中，我们通过实证证明，通过使用领域内数据对人工构建的提取提示的前缀进行基于数据的引导，可以将PII的可提取性提高十倍以上。我们的方法PII-Compass，在使用1、128和2308次查询时，分别实现了0.92%、3.9%和6.86%的电话号码提取率，即每15人中就有1人的电话号码可被提取。