Large Language Models (LLMs) could enhance access to the legal system. However, empirical research on their effectiveness in conducting legal tasks is scant. We study securities cases involving cryptocurrencies as one of numerous contexts where AI could support the legal process, studying LLMs' legal reasoning and drafting capabilities. We examine whether a) an LLM can accurately determine which laws are potentially being violated from a fact pattern, and b) whether there is a difference in juror decision-making based on complaints written by a lawyer compared to an LLM. We feed fact patterns from real-life cases to GPT-3.5 and evaluate its ability to determine correct potential violations from the scenario and exclude spurious violations. Second, we had mock jurors assess complaints written by the LLM and lawyers. GPT-3.5's legal reasoning skills proved weak, though we expect improvement in future models, particularly given the violations it suggested tended to be correct (it merely missed additional, correct violations). GPT-3.5 performed better at legal drafting, and jurors' decisions were not statistically significantly associated with the author of the document upon which they based their decisions. Because LLMs cannot satisfactorily conduct legal reasoning tasks, they would be unable to replace lawyers at this stage. However, their drafting skills (though, perhaps, still inferior to lawyers), could provide access to justice for more individuals by reducing the cost of legal services. Our research is the first to systematically study LLMs' legal drafting and reasoning capabilities in litigation, as well as in securities law and cryptocurrency-related misconduct.

翻译：大型语言模型（LLM）有望提升法律系统的可及性，但关于其在执行法律任务方面有效性的实证研究尚显不足。本研究聚焦加密货币相关证券案件——这是人工智能可能助力法律程序的众多场景之一，系统考察LLM的法律推理与法律起草能力。我们探究两个问题：a) LLM能否基于事实模式准确判断潜在违反的法律条款；b) 基于律师或LLM撰写的诉状，陪审员决策是否存在差异。通过将真实案例的事实模式输入GPT-3.5，我们评估其从场景中识别正确潜在违法条款并排除无关条款的能力。其次，我们邀请模拟陪审员评估由LLM和律师撰写的诉状。实验表明，GPT-3.5的法律推理能力较弱（尽管其建议的违法条款往往正确，但会遗漏其他正确违法条款），我们预期未来模型将有所改进。GPT-3.5在法律起草方面表现更佳，陪审员决策与诉状作者身份之间无统计显著关联。由于LLM尚无法令人满意地完成法律推理任务，当前阶段难以替代律师。但其起草能力（尽管可能仍逊于律师）可通过降低法律服务成本，为更多人提供司法可及性。本研究是首个系统评估LLM在诉讼、证券法及加密货币相关不当行为中法律起草与推理能力的实证研究。