Large Language Models (LLMs) could enhance access to the legal system. However, empirical research on their effectiveness in conducting legal tasks is scant. We study securities cases involving cryptocurrencies as one of numerous contexts where AI could support the legal process, studying LLMs' legal reasoning and drafting capabilities. We examine whether a) an LLM can accurately determine which laws are potentially being violated from a fact pattern, and b) whether there is a difference in juror decision-making based on complaints written by a lawyer compared to an LLM. We feed fact patterns from real-life cases to GPT-3.5 and evaluate its ability to determine correct potential violations from the scenario and exclude spurious violations. Second, we had mock jurors assess complaints written by the LLM and lawyers. GPT-3.5's legal reasoning skills proved weak, though we expect improvement in future models, particularly given the violations it suggested tended to be correct (it merely missed additional, correct violations). GPT-3.5 performed better at legal drafting, and jurors' decisions were not statistically significantly associated with the author of the document upon which they based their decisions. Because LLMs cannot satisfactorily conduct legal reasoning tasks, they would be unable to replace lawyers at this stage. However, their drafting skills (though, perhaps, still inferior to lawyers), could provide access to justice for more individuals by reducing the cost of legal services. Our research is the first to systematically study LLMs' legal drafting and reasoning capabilities in litigation, as well as in securities law and cryptocurrency-related misconduct.

翻译：大型语言模型（LLMs）有望提升法律系统的可及性。然而，关于其在法律任务中有效性的实证研究尚显不足。我们以涉及加密货币的证券案件为例，研究人工智能在法律流程中的潜在支持作用，聚焦LLMs的法律推理与起草能力。我们探讨以下问题：（a）LLM能否根据事实模式准确判断可能违反的法律条款；（b）基于律师与LLM撰写的起诉状，陪审团决策是否存在差异。我们向GPT-3.5输入真实案例的事实模式，评估其从场景中识别正确潜在违法行为并排除无关违法行为的能力。其次，我们要求模拟陪审员评估由LLM和律师撰写的起诉状。结果表明，GPT-3.5的法律推理能力较弱——尽管我们预期未来模型会有所改进，因其提出的违法行为建议多为正确（仅遗漏了额外的正确违法行为）。GPT-3.5在法律起草方面表现更优，陪审员决策与其所依据文件作者之间并无统计显著性关联。由于LLMs无法令人满意地完成法律推理任务，现阶段尚不能取代律师。然而，其起草能力（尽管可能仍逊于律师）可通过降低法律服务成本，使更多人获得司法救济。本研究首次系统性地考察了LLMs在诉讼、证券法及加密货币相关不当行为中的法律起草与推理能力。