Legal intake, the process of finding out if an applicant is eligible for help from a free legal aid program, takes significant time and resources. In part this is because eligibility criteria are nuanced, open-textured, and require frequent revision as grants start and end. In this paper, we investigate the use of large language models (LLMs) to reduce this burden. We describe a digital intake platform that combines logical rules with LLMs to offer eligibility recommendations, and we evaluate the ability of 8 different LLMs to perform this task. We find promising results for this approach to help close the access to justice gap, with the best model reaching an F1 score of .82, while minimizing false negatives.
翻译:法律案件受理,即确定申请人是否符合免费法律援助项目资格的过程,需要耗费大量时间和资源。这部分是因为资格标准具有细微差别、开放性特征,且随着资助项目的启动和结束需要频繁修订。本文研究了利用大型语言模型(LLMs)来减轻这一负担的方法。我们描述了一个结合逻辑规则与LLMs的数字受理平台,该平台可提供资格建议,并评估了8种不同LLM执行此任务的能力。我们发现该方法在帮助缩小司法可及性差距方面展现出良好前景,最佳模型的F1分数达到0.82,同时最大限度地减少了假阴性案例。