Large language models have exhibited robust performance across diverse natural language processing tasks. This report introduces TechGPT-2.0, a project designed to enhance the capabilities of large language models specifically in knowledge graph construction tasks, including named entity recognition (NER) and relationship triple extraction (RTE) tasks in NLP applications. Additionally, it serves as a LLM accessible for research within the Chinese open-source model community. We offer two 7B large language model weights and a QLoRA weight specialized for processing lengthy texts.Notably, TechGPT-2.0 is trained on Huawei's Ascend server. Inheriting all functionalities from TechGPT-1.0, it exhibits robust text processing capabilities, particularly in the domains of medicine and law. Furthermore, we introduce new capabilities to the model, enabling it to process texts in various domains such as geographical areas, transportation, organizations, literary works, biology, natural sciences, astronomical objects, and architecture. These enhancements also fortified the model's adeptness in handling hallucinations, unanswerable queries, and lengthy texts. This report provides a comprehensive and detailed introduction to the full fine-tuning process on Huawei's Ascend servers, encompassing experiences in Ascend server debugging, instruction fine-tuning data processing, and model training. Our code is available at https://github.com/neukg/TechGPT-2.0

翻译：大语言模型在各种自然语言处理任务中展现出强大的性能。本报告介绍了TechGPT-2.0项目，该项目旨在增强大语言模型在知识图谱构建任务中的能力，包括自然语言处理应用中的命名实体识别和关系三元组抽取任务。此外，该项目还为中国开源模型社区提供了一个可供研究使用的大语言模型。我们提供了两个7B规模的大语言模型权重，以及一个专门用于处理长文本的QLoRA权重。需要特别指出的是，TechGPT-2.0在华为昇腾服务器上完成训练。该模型继承了TechGPT-1.0的全部功能，在医学和法律领域展现出强大的文本处理能力。此外，我们为模型引入了新功能，使其能够处理地理区域、交通、组织机构、文学作品、生物学、自然科学、天体对象和建筑等多个领域的文本。这些增强还提升了模型在处理幻觉、不可回答问题以及长文本方面的能力。本报告全面详细地介绍了在华为昇腾服务器上进行的全微调过程，涵盖昇腾服务器调试、指令微调数据处理以及模型训练等方面的经验。我们的代码已开源至 https://github.com/neukg/TechGPT-2.0。