Predicting legal judgments with reliable confidence is paramount for responsible legal AI applications. While transformer-based deep neural networks (DNNs) like BERT have demonstrated promise in legal tasks, accurately assessing their prediction confidence remains crucial. We present a novel Bayesian approach called BayesJudge that harnesses the synergy between deep learning and deep Gaussian Processes to quantify uncertainty through Bayesian kernel Monte Carlo dropout. Our method leverages informative priors and flexible data modelling via kernels, surpassing existing methods in both predictive accuracy and confidence estimation as indicated through brier score. Extensive evaluations of public legal datasets showcase our model's superior performance across diverse tasks. We also introduce an optimal solution to automate the scrutiny of unreliable predictions, resulting in a significant increase in the accuracy of the model's predictions by up to 27\%. By empowering judges and legal professionals with more reliable information, our work paves the way for trustworthy and transparent legal AI applications that facilitate informed decisions grounded in both knowledge and quantified uncertainty.

翻译：摘要：在可靠置信度下进行法律判决预测，对于负责任的司法人工智能应用至关重要。尽管基于Transformer的深度神经网络（如BERT）在法律任务中展现出潜力，但准确评估其预测置信度仍具有关键意义。我们提出了一种名为BayesJudge的新型贝叶斯方法，通过贝叶斯核蒙特卡洛丢弃法协同利用深度学习与深度高斯过程的优势来量化不确定性。该方法通过核函数引入信息性先验与灵活的数据建模，在布里尔分数指标上，其预测精度与置信度估计均超越现有方法。在公开法律数据集上的全面评估表明，本模型在不同任务中均展现出卓越性能。我们还提出了一种最优方案来自动审查不可靠预测，使模型预测准确率最高提升27%。通过为法官及法律从业者提供更可靠的信息，本研究为构建可信且透明的法律人工智能应用铺平了道路，助力基于知识积累与量化不确定性的知情决策。