In recent years, large language models have attracted significant attention due to their exceptional performance across a multitude of natural language process tasks, and have been widely applied in various fields. However, the application of large language models in the Intellectual Property (IP) space is challenging due to the strong need for specialized knowledge, privacy protection, processing of extremely long text in this field. In this technical report, we present for the first time a low-cost, standardized procedure for training IP-oriented LLMs, meeting the unique requirements of the IP domain. Using this standard process, we have trained the PatentGPT series models based on open-source pretrained models. By evaluating them on the open-source IP-oriented benchmark MOZIP, our domain-specific LLMs outperforms GPT-4, indicating the effectiveness of the proposed training procedure and the expertise of the PatentGPT models in the IP demain. What is impressive is that our model significantly outperformed GPT-4 on the 2019 China Patent Agent Qualification Examination by achieving a score of 65, reaching the level of human experts. Additionally, the PatentGPT model, which utilizes the SMoE architecture, achieves performance comparable to that of GPT-4 in the IP domain and demonstrates a better cost-performance ratio on long-text tasks, potentially serving as an alternative to GPT-4 within the IP domain.

翻译：近年来，大语言模型因其在众多自然语言处理任务中的卓越表现而备受关注，并已广泛应用于各个领域。然而，大语言模型在知识产权领域的应用面临挑战，这主要源于该领域对专业知识、隐私保护及超长文本处理的强烈需求。本技术报告首次提出了一种低成本、标准化的面向知识产权的大语言模型训练流程，以满足知识产权领域的独特需求。通过该标准化流程，我们基于开源预训练模型训练了PatentGPT系列模型。在开源知识产权领域基准测试MOZIP上的评估结果显示，我们的领域专用大语言模型性能优于GPT-4，这验证了所提出训练流程的有效性以及PatentGPT模型在知识产权领域的专业性。令人瞩目的是，我们的模型在2019年中国专利代理人资格考试中获得65分，显著超越GPT-4，达到了人类专家水平。此外，采用SMoE架构的PatentGPT模型在知识产权领域实现了与GPT-4相当的性能，并在长文本任务中展现出更优的性价比，有望成为知识产权领域内GPT-4的替代方案。