Large Language Models (LLMs) could enhance access to the legal system. However, empirical research on their effectiveness in conducting legal tasks is scant. We study securities cases involving cryptocurrencies as one of numerous contexts where AI could support the legal process, studying LLMs' legal reasoning and drafting capabilities. We examine whether a) an LLM can accurately determine which laws are potentially being violated from a fact pattern, and b) whether there is a difference in juror decision-making based on complaints written by a lawyer compared to an LLM. We feed fact patterns from real-life cases to GPT-3.5 and evaluate its ability to determine correct potential violations from the scenario and exclude spurious violations. Second, we had mock jurors assess complaints written by the LLM and lawyers. GPT-3.5's legal reasoning skills proved weak, though we expect improvement in future models, particularly given the violations it suggested tended to be correct (it merely missed additional, correct violations). GPT-3.5 performed better at legal drafting, and jurors' decisions were not statistically significantly associated with the author of the document upon which they based their decisions. Because LLMs cannot satisfactorily conduct legal reasoning tasks, they would be unable to replace lawyers at this stage. However, their drafting skills (though, perhaps, still inferior to lawyers), could provide access to justice for more individuals by reducing the cost of legal services. Our research is the first to systematically study LLMs' legal drafting and reasoning capabilities in litigation, as well as in securities law and cryptocurrency-related misconduct.

翻译：大型语言模型（LLM）可能增强法律系统的可及性。然而，关于其在执行法律任务方面的有效性的实证研究仍然匮乏。我们以涉及加密货币的证券案件为背景，研究人工智能在支持法律程序中的多种可能性，重点考察LLM的法律推理与文书起草能力。我们探究：(a) LLM能否根据案情事实准确判断可能违反的法律条款；(b) 基于律师或LLM撰写的起诉状，陪审员的决策是否存在差异。我们将真实案例的案情事实输入GPT-3.5，评估其从情境中识别正确潜在违法行为并排除无关违法行为的能力。其次，我们让模拟陪审员评估由LLM和律师撰写的起诉状。结果显示，GPT-3.5的法律推理能力薄弱，但我们预计未来模型将有所改进——尤其是其建议的违法行为多属正确（仅是遗漏了额外的正确违法行为）。GPT-3.5在法律起草方面表现更优，而陪审员的决策与其所依据文书的作者之间并无统计学显著关联。由于LLM无法令人满意地完成法律推理任务，现阶段尚不能取代律师。然而，其起草能力（尽管可能仍逊于律师）可通过降低法律服务成本，使更多个体获得正义。本研究首次系统评估了LLM在诉讼中的法律起草与推理能力，以及其在证券法和加密货币相关不当行为领域的应用。