Many individuals are likely to face a legal dispute at some point in their lives, but their lack of understanding of how to navigate these complex issues often renders them vulnerable. The advancement of natural language processing opens new avenues for bridging this legal literacy gap through the development of automated legal aid systems. However, existing legal question answering (LQA) approaches often suffer from a narrow scope, being either confined to specific legal domains or limited to brief, uninformative responses. In this work, we propose an end-to-end methodology designed to generate long-form answers to any statutory law questions, utilizing a "retrieve-then-read" pipeline. To support this approach, we introduce and release the Long-form Legal Question Answering (LLeQA) dataset, comprising 1,868 expert-annotated legal questions in the French language, complete with detailed answers rooted in pertinent legal provisions. Our experimental results demonstrate promising performance on automatic evaluation metrics, but a qualitative analysis uncovers areas for refinement. As one of the only comprehensive, expert-annotated long-form LQA dataset, LLeQA has the potential to not only accelerate research towards resolving a significant real-world issue, but also act as a rigorous benchmark for evaluating NLP models in specialized domains. We publicly release our code, data, and models.
翻译:许多人一生中可能面临法律纠纷,但由于缺乏应对这些复杂问题的知识,他们往往处于弱势地位。自然语言处理的进步为开发自动化法律援助系统、弥合法律素养差距开辟了新途径。然而,现有法律问答方法常存在范围狭窄的问题,要么局限于特定法律领域,要么仅生成简短且信息有限的回复。本研究提出一种端到端方法,通过“检索-阅读”流水线生成针对任何成文法问题的详尽答案。为支撑该方法,我们创建并发布了法语长篇法律问答数据集,包含1,868个由专家标注的法律问题及其基于相关法律条款的详细答案。实验结果显示,该模型在自动评估指标上表现良好,但定性分析揭示了仍需改进之处。作为少数全面的、包含专家标注的长篇法律问答数据集之一,该数据集不仅有望加速解决这一重要现实问题,还可作为评估专业领域自然语言处理模型的严格基准。我们已公开发布代码、数据和模型。