Large Language Models (LLMs) have shown the potential to revolutionize natural language processing tasks in various domains, sparking great interest in vertical-specific large models. However, unlike proprietary models such as BloombergGPT and FinGPT, which have leveraged their unique data accumulations to make strides in the finance domain, there hasn't not many similar large language models in the Chinese legal domain to facilitate its digital transformation. In this paper, we propose an open-source legal large language model named ChatLaw. Due to the importance of data quality, we carefully designed a legal domain fine-tuning dataset. Additionally, to overcome the problem of model hallucinations in legal data screening during reference data retrieval, we introduce a method that combines vector database retrieval with keyword retrieval to effectively reduce the inaccuracy of relying solely on vector database retrieval. Furthermore, we propose a self-attention method to enhance the ability of large models to overcome errors present in reference data, further optimizing the issue of model hallucinations at the model level and improving the problem-solving capabilities of large models. We also open-sourced our model and part of the data at https://github.com/PKU-YuanGroup/ChatLaw.

翻译：大语言模型在多个领域展现出革新自然语言处理任务的潜力，激发了垂直领域专用大模型的浓厚兴趣。然而，与BloombergGPT、FinGPT等利用独特数据积累在金融领域取得突破的专有模型不同，中文法律领域尚缺乏类似的大语言模型以推动其数字化转型。本文提出名为ChatLaw的开源法律大语言模型。鉴于数据质量的重要性，我们精心设计了一个法律领域的微调数据集。此外，为克服参考数据检索过程中法律数据筛选的模型幻觉问题，我们引入了一种结合向量数据库检索与关键词检索的方法，有效减少了仅依赖向量数据库检索的不准确性。进一步地，我们提出一种自注意力方法，以增强大模型克服参考数据中错误的能-力，从而在模型层面优化模型幻觉问题，提升大模型的问题解决能力。我们已在https://github.com/PKU-YuanGroup/ChatLaw开源了模型及部分数据。