When applying LLMs to real-world enterprise operations, LLMs need to handle proprietary knowledge in small domains of specific operations ($\textbf{micro domains}$). A previous study shows micro domain-adaptive pre-training ($\textbf{mDAPT}$) with fewer documents is effective, similarly to DAPT in larger domains. However, it evaluates mDAPT only on multiple-choice questions; thus, its effectiveness for generative tasks in real-world operations remains unknown. We aim to reveal the potential and bottlenecks of mDAPT for generative tasks. To this end, we disentangle the answering process into three subtasks and evaluate the performance of each subtask: (1) $\textbf{eliciting}$ facts relevant to questions from an LLM's own knowledge, (2) $\textbf{reasoning}$ over the facts to obtain conclusions, and (3) $\textbf{composing}$ long-form answers based on the conclusions. We verified mDAPT on proprietary IT product knowledge for real-world questions in IT technical support operations. As a result, mDAPT resolved the elicitation task that the base model struggled with but did not resolve other subtasks. This clarifies mDAPT's effectiveness in the knowledge aspect and its bottlenecks in other aspects. Further analysis empirically shows that resolving the elicitation and reasoning tasks ensures sufficient performance (over 90%), emphasizing the need to enhance reasoning capability.

翻译：将大语言模型应用于现实企业运营时，模型需要处理特定运营小领域中的专有知识（$\textbf{微领域}$）。先前研究表明，使用较少文档进行的微领域自适应预训练（$\textbf{mDAPT}$）是有效的，其效果类似于更大领域的DAPT。然而，该研究仅通过选择题评估mDAPT，因此其对实际运营中生成任务的有效性仍属未知。本研究旨在揭示mDAPT在生成任务中的潜力与瓶颈。为此，我们将回答过程解构为三个子任务并分别评估其性能：（1）从大语言模型自身知识中$\textbf{提取}$与问题相关的事实，（2）基于事实进行$\textbf{推理}$以获得结论，（3）根据结论$\textbf{组织}$长篇回答。我们在IT技术支持运营场景中，针对专有IT产品知识的实际问题验证了mDAPT。结果表明，mDAPT解决了基础模型难以处理的提取任务，但未能解决其他子任务。这明确了mDAPT在知识层面的有效性及其在其他方面的瓶颈。进一步分析通过实证表明，解决提取与推理任务可确保足够性能（超过90%），这凸显了增强推理能力的必要性。