While large language models (LLMs) are increasingly used to summarize long documents, this trend poses significant challenges in the legal domain, where the factual accuracy of deposition summaries is crucial. Nugget-based methods have been shown to be extremely helpful for the automated evaluation of summarization approaches. In this work, we translate these methods to the user side and explore how nuggets could directly assist end users. Although prior systems have demonstrated the promise of nugget-based evaluation, its potential to support end users remains underexplored. Focusing on the legal domain, we present a prototype that leverages a factual nugget-based approach to support legal professionals in two concrete scenarios: (1) determining which of two summaries is better, and (2) manually improving an automatically generated summary.
翻译:尽管大型语言模型(LLMs)越来越多地用于总结长篇文档,但这一趋势在法律领域带来了重大挑战,因为证词摘要的事实准确性至关重要。基于信息块的方法已被证明对摘要方法的自动化评估极为有效。在本研究中,我们将这些方法迁移至用户端,探索信息块如何直接辅助终端用户。尽管先前系统已展示了基于信息块的评估方法的潜力,但其支持终端用户的能力仍未得到充分探索。聚焦于法律领域,我们提出了一个原型系统,该系统利用基于事实信息块的方法,在两个具体场景中支持法律专业人员:(1)判断两个摘要中哪一个更优,以及(2)手动改进自动生成的摘要。