In recent years, large language models(LLMs) have attracted significant attention due to their exceptional performance across a multitude of natural language process tasks, and have been widely applied in various fields. However, the application of large language models in the Intellectual Property (IP) domain is challenging due to the strong need for specialized knowledge, privacy protection, processing of extremely long text in this field. In this technical report, we present for the first time a low-cost, standardized procedure for training IP-oriented LLMs, meeting the unique requirements of the IP domain. Using this standard process, we have trained the PatentGPT series models based on open-source pretrained models. By evaluating them on the open-source IP-oriented benchmark MOZIP, our domain-specific LLMs outperforms GPT-4, indicating the effectiveness of the proposed training procedure and the expertise of the PatentGPT models in the IP domain. Remarkably, our model surpassed GPT-4 on the 2019 China Patent Agent Qualification Examination, scoring 65 and matching human expert levels. Additionally, the PatentGPT model, which utilizes the SMoE architecture, achieves performance comparable to that of GPT-4 in the IP domain and demonstrates a better cost-performance ratio on long-text tasks, potentially serving as an alternative to GPT-4 within the IP domain.

翻译：近年来，大语言模型（LLMs）凭借其在众多自然语言处理任务中的卓越表现引起了广泛关注，并被广泛应用于各个领域。然而，大语言模型在知识产权领域的应用面临挑战，原因在于该领域对专业知识、隐私保护以及超长文本处理有着强烈需求。本技术报告首次提出一种低成本、标准化的面向知识产权的大语言模型训练流程，以满足IP领域的独特需求。通过这一标准化流程，我们基于开源预训练模型训练了PatentGPT系列模型。在开源IP领域基准测试MOZIP上的评估表明，我们的领域专用大语言模型超越了GPT-4，验证了所提训练流程的有效性以及PatentGPT模型在IP领域的专业能力。值得注意的是，我们的模型在2019年中国专利代理人资格考试中超越GPT-4，取得65分，达到人类专家水平。此外，采用SMoE架构的PatentGPT模型在IP领域实现了与GPT-4相当的性能，并在长文本任务中展现出更优的性价比，有望成为IP领域内GPT-4的替代方案。