Understanding and engaging with privacy policies is crucial for online privacy, yet these documents remain notoriously complex and difficult to navigate. We present PRISMe, an interactive browser extension that combines LLM-based policy assessment with a dashboard and customizable chat interface, enabling users to skim quick overviews or explore policy details in depth while browsing. We conduct a user study (N=22) with participants of diverse privacy knowledge to investigate how users interpret the tool's explanations and how it shapes their engagement with privacy policies, identifying distinct interaction patterns. Participants valued the clear overviews and conversational depth, but flagged some issues, particularly adversarial robustness and hallucination risks. Thus, we investigate how a retrieval-augmented generation (RAG) approach can alleviate issues by re-running the chat queries from the study. Our findings surface design challenges as well as technical trade-offs, contributing actionable insights for developing future user-centered, trustworthy privacy policy analysis tools.

翻译：理解和处理隐私政策对于在线隐私保护至关重要，但这些文件素以复杂难懂著称。我们提出了PRISMe——一款结合基于LLM的政策评估功能、可视化仪表板与可定制聊天界面的交互式浏览器扩展，使用户在浏览过程中既能快速概览政策要点，又能深入探究具体条款。我们开展了一项用户研究（N=22），邀请具有不同隐私知识背景的参与者，探究用户如何解读工具生成的解释说明，以及该工具如何影响他们与隐私政策的互动模式，并识别出显著的交互行为特征。参与者高度评价了该工具的清晰概览功能和对话式深度解析能力，但也指出若干问题，特别是对抗鲁棒性和幻觉风险。为此，我们通过重新运行研究中的聊天查询，探究检索增强生成（RAG）方法如何缓解这些问题。我们的研究结果揭示了设计层面的挑战与技术取舍，为开发未来以用户为中心、可信赖的隐私政策分析工具提供了可操作的见解。