Probabilistic reasoning is a key aspect of both human and artificial intelligence that allows for handling uncertainty and ambiguity in decision-making. In this paper, we introduce a new numerical reasoning task under uncertainty for large language models, focusing on estimating the privacy risk of user-generated documents containing privacy-sensitive information. We propose BRANCH, a new LLM methodology that estimates the k-privacy value of a text-the size of the population matching the given information. BRANCH factorizes a joint probability distribution of personal information as random variables. The probability of each factor in a population is estimated separately using a Bayesian network and combined to compute the final k-value. Our experiments show that this method successfully estimates the k-value 73% of the time, a 13% increase compared to o3-mini with chain-of-thought reasoning. We also find that LLM uncertainty is a good indicator for accuracy, as high-variance predictions are 37.47% less accurate on average.

翻译：概率推理是人类与人工智能处理决策中不确定性与模糊性的关键能力。本文针对大型语言模型提出一种新的不确定性下的数值推理任务，重点评估包含隐私敏感信息的用户生成文档的隐私风险。我们提出BRANCH这一新的LLM方法，用于估算文本的k-隐私值——即与给定信息匹配的群体规模。BRANCH将个人信息的联合概率分布分解为随机变量，通过贝叶斯网络分别估计各因子在群体中的概率，并组合计算最终的k值。实验表明，该方法成功估算k值的准确率达到73%，较采用思维链推理的o3-mini模型提升13%。研究还发现，LLM预测的不确定性可作为准确性的有效指标，高方差预测的平均准确率降低37.47%。