Large language models have demonstrated impressive capabilities across various domains. However, their general-purpose nature often limits their effectiveness in specialized fields such as energy, where deep technical expertise and precise domain knowledge are essential. In this paper, we introduce EnergyGPT, a domain-specialized language model tailored for the energy sector, developed by fine-tuning the LLaMA 3.1-8B model on a high-quality, curated corpus of energy-related texts. We consider two adaptation strategies: a full-parameter Supervised Fine-Tuning variant and a parameter-efficient LoRA-based variant that updates only a small fraction of the model parameters. We present a complete development pipeline, including data collection and curation, model fine-tuning, benchmark design and LLM-judge choice, evaluation, and deployment. Through this work, we demonstrate that our training strategy enables improvements in domain relevance and performance without the need for large-scale infrastructure. By evaluating the performance of both EnergyGPT variants using domain-specific question-answering benchmarks, our results show that the adapted models consistently outperform the base model in most energy-related language understanding and generation tasks, with the LoRA variant achieving competitive gains at significantly reduced training cost.

翻译：大型语言模型已在多个领域展现出卓越能力。然而，其通用性特征往往限制了在能源等深度专业领域的应用效果——这类领域需要深厚的技术专长和精确的领域知识。本文提出EnergyGPT，一种通过在高品质能源领域精选语料库上微调LLaMA 3.1-8B模型而开发的领域专用语言模型。我们采用两种适配策略：全参数监督微调变体与仅更新少量模型参数的参数高效型LoRA变体。我们呈现了完整的开发流程，包括数据采集与筛选、模型微调、基准测试设计与大语言模型评估选择、模型评估及部署。研究表明，本训练策略能在无需大规模基础设施的条件下提升领域相关性与模型性能。通过使用领域特定问答基准测试对两种EnergyGPT变体进行评估，结果显示适配模型在大多数能源相关的语言理解与生成任务中均优于基础模型，其中LoRA变体在显著降低训练成本的同时仍能获得具有竞争力的性能提升。