Identifying contextual integrity (CI) and governing knowledge commons (GKC) parameters in privacy policy texts can facilitate normative privacy analysis. However, GKC-CI annotation has heretofore required manual or crowdsourced effort. This paper demonstrates that high-accuracy GKC-CI parameter annotation of privacy policies can be performed automatically using large language models. We fine-tune 18 open-source and proprietary models on 21,588 GKC-CI annotations from 16 ground truth privacy policies. Our best-performing model (fine-tuned GPT-3.5 Turbo with prompt engineering) has an accuracy of 86%, exceeding the performance of prior crowdsourcing approaches despite the complexity of privacy policy texts and the nuance of the GKC-CI annotation task. We apply our best-performing model to privacy policies from 164 popular online services, demonstrating the effectiveness of scaling GKC-CI annotation for data exploration. We make all annotated policies as well as the training data and scripts needed to fine-tune our best-performing model publicly available for future research.

翻译：在隐私政策文本中识别情境完整性（CI）与治理知识共享（GKC）参数可促进规范性隐私分析，但此前GKC-CI标注工作需依赖人工或众包完成。本文证明，利用大语言模型可自动实现隐私政策的高精度GKC-CI参数标注。我们基于16份真实隐私政策的21,588条GKC-CI标注数据，对18个开源与专有模型进行微调。表现最优的模型（经提示工程微调的GPT-3.5 Turbo）准确率达86%，尽管隐私政策文本复杂且GKC-CI标注任务具有细微差异，该性能仍超越此前众包方法。我们将最优模型应用于164个主流在线服务的隐私政策，验证了规模化GKC-CI标注对数据探索的有效性。所有标注政策数据、训练数据及微调最优模型所需脚本均已公开供未来研究使用。