Environmental, Social, and Governance (ESG) considerations play a central role in contemporary financial decision-making. In parallel, Large Language Model (LLM) applications in this domain have primarily emphasized well-defined discriminative tasks, such as classification or scoring, which have proven effective for structured analysis and benchmarking. However, this prevailing focus offers limited support for more interactive and generative ESG scenarios, where embedded domain knowledge and contextual understanding are essential. In this work, we propose an ESG-oriented adaptation pipeline for LLMs that integrates ESG principles not only as a target domain, but also as guiding constraints throughout training and evaluation. Building on the Qwen-3-4B architecture, we explore parameter-efficient adaptation strategies using Low-Rank Adaptation (LoRA) and the Instruction-Residual Method (IRM) to produce three ESG-specialized models. We evaluate the proposed models on ESG question answering under both zero-shot and knowledge-augmented settings, using a diverse set of generative, semantic, readability, and environmental impact metrics. Our results show that the ESG-adapted models consistently outperform their original counterparts and competitive baselines such as Llama-3 and Gemma-3. Although limitations remain in tool-based knowledge integration, this work establishes a foundation for ESG-oriented language generation and highlights the importance of responsible, domain-aware LLM adaptation.

翻译：环境、社会和治理（ESG）考量在当代金融决策中扮演核心角色。与此同时，大语言模型（LLM）在该领域的应用主要侧重于定义明确的判别任务（如分类或评分），这些方法已被证明对结构化分析与基准测试有效。然而，这种主流关注方式对更具交互性与生成性的ESG场景支持有限，而此类场景恰恰需要嵌入领域知识与上下文理解能力。本文提出一种面向ESG的LLM适配流程，该流程不仅将ESG原则作为目标领域，更将其作为贯穿训练与评估全过程的指导性约束。基于Qwen-3-4B架构，我们探索了采用低秩适应（LoRA）和指令残差法（IRM）的参数高效适配策略，从而构建出三个ESG专用模型。在零样本与知识增强两种设定下，我们通过涵盖生成质量、语义准确性、可读性及环境影响的多元指标，对模型进行ESG问答评估。结果表明，ESG适配模型始终优于原始模型及Llama-3、Gemma-3等竞争基线。尽管在基于工具的知识整合方面仍存局限，本研究为ESG导向的语言生成奠定了基础，并突显了负责任且具领域感知能力的LLM适配的重要性。