Large language models (LLMs) are widely used to fulfill users' information needs; users ask LLMs about the weather, pose educational questions, and consult them for legal assistance. One particularly understudied area is digital security and privacy (S&P), where users may seek LLMs' help on how to secure their online accounts or protect their computers from cyber attacks. To the best of our knowledge, no prior study has collected or analyzed the S&P questions users ask LLMs; prior research on LLM response quality relied on expert-authored S&P misconceptions or FAQs rather than user queries. Drawing from WildChat, a dataset of 3.2M user-LLM conversations collected in the wild, our study identifies 14,727 S&P prompts and categorizes them into nine categories covering a wide range of S&P topics. From the S&P prompts, we sampled 450 and performed a thematic analysis to characterize the S&P questions users ask LLMs. Separate from the thematic analysis, we curated 270 advice-seeking S&P prompts, where users ask for recommendations, guidance, or specific S&P information. We measured LLM response quality and consistency when posing the prompt to LLMs 10 times. We found that commercial LLMs outperform open-weight models (GPT 5.5 provided "good enough" responses on 98% of prompts; Llama 4 on 47%). However, among prompts that received high-quality responses on average, commercial models sometimes produce contradictory responses across runs, risking confusing or misleading users.

翻译：大语言模型（LLMs）被广泛用于满足用户的信息需求；用户向LLMs询问天气、提出教育问题、寻求法律咨询等。其中一个特别研究不足的领域是数字安全与隐私（S&P），用户可能在此领域寻求LLMs的帮助，例如如何保护在线账户或保护计算机免受网络攻击。据我们所知，尚无先例研究收集或分析用户向LLMs提出的S&P问题；先前关于LLM响应质量的研究依赖于专家撰写的S&P误解或常见问题解答，而非用户真实查询。基于WildChat（一个在现实环境中收集的包含320万用户-LLM对话的数据集），我们的研究识别出14,727条S&P提示，并将其分类为涵盖广泛S&P主题的九个类别。从这些S&P提示中，我们抽样了450条，并进行了主题分析，以刻画用户向LLMs提出的S&P问题特征。与主题分析并行，我们还整理了270条寻求建议的S&P提示，其中用户请求建议、指导或特定的S&P信息。我们通过将同一提示多次（10次）提交给LLMs，衡量了LLM的响应质量与一致性。结果发现，商业LLMs优于开放权重模型（GPT 5.5在98%的提示上提供了“足够好”的响应；Llama 4为47%）。然而，在平均获得高质量响应的提示中，商业模型有时在不同运行中产生矛盾的响应，存在误导用户的潜在风险。