In recent years, large language models have attracted significant attention due to their exceptional performance across a multitude of natural language process tasks, and have been widely applied in various fields. However, the application of large language models in the Intellectual Property (IP) space is challenging due to the strong need for specialized knowledge, privacy protection, processing of extremely long text in this field. In this technical report, we present for the first time a low-cost, standardized procedure for training IP-oriented LLMs, meeting the unique requirements of the IP domain. Using this standard process, we have trained the PatentGPT series models based on open-source pretrained models. By evaluating them on the open-source IP-oriented benchmark MOZIP, our domain-specific LLMs outperforms GPT-4, indicating the effectiveness of the proposed training procedure and the expertise of the PatentGPT models in the IP demain. What is impressive is that our model significantly outperformed GPT-4 on the 2019 China Patent Agent Qualification Examination by achieving a score of 65, reaching the level of human experts. Additionally, the PatentGPT model, which utilizes the SMoE architecture, achieves performance comparable to that of GPT-4 in the IP domain and demonstrates a better cost-performance ratio on long-text tasks, potentially serving as an alternative to GPT-4 within the IP domain.

翻译：近年来，大型语言模型因其在众多自然语言处理任务中的卓越表现而备受关注，并已广泛应用于各个领域。然而，大型语言模型在知识产权（IP）领域的应用面临挑战，主要源于该领域对专业知识、隐私保护以及超长文本处理能力的强烈需求。本技术报告首次提出一种面向IP的低成本、标准化的大语言模型训练流程，以满足知识产权领域的独特需求。基于该标准流程，我们利用开源预训练模型训练了PatentGPT系列模型。通过在开源IP专项基准测试集MOZIP上的评估，我们的领域专用大语言模型性能超越GPT-4，这证明了所提训练流程的有效性以及PatentGPT模型在IP领域的专业性。令人瞩目的是，我们的模型在2019年全国专利代理师资格考试中以65分的成绩显著超越GPT-4，达到人类专家水平。此外，采用SMoE架构的PatentGPT模型在IP领域实现了与GPT-4相媲美的性能，并在长文本任务中展现出更优的性价比，有望成为IP领域内GPT-4的替代方案。