We conduct the first empirical study on using knowledge transfer to improve the generalization ability of large language models (LLMs) in software engineering tasks, which often require LLMs to generalize beyond their training data. Our proposed general knowledge transfer approach guides the LLM towards a similar and familiar API or code snippet it has encountered before, improving the model's generalization ability for unseen knowledge. We apply this approach to three software engineering tasks: API inference, code example generation, and FQN inference, and find transfer span, transfer strategy, and transfer architecture as key factors affecting the method. Our findings demonstrate the feasibility of knowledge transfer and its potential to enhance LLMs' performance in various software engineering tasks. The effectiveness of knowledge transfer varies depending on the target domain and task, with the hierarchical strategy being more effective than direct transfer, and AI-Chain outperforming CoT in prompt design. The implications of these findings extend beyond software engineering tasks and suggest that knowledge transfer can enhance LLMs' ability to handle unknowns in any natural language task.

翻译：我们首次通过实证研究探讨了利用知识迁移提升大语言模型（LLMs）在软件工程任务中泛化能力的可行性——此类任务常要求LLMs超越训练数据范围进行泛化。我们提出的通用知识迁移方法引导LLM回溯其曾接触过的相似或熟悉的API及代码片段，从而提升模型对未知知识的泛化能力。我们将该方法应用于三项软件工程任务：API推断、代码示例生成和全限定名（FQN）推断，并发现迁移跨度、迁移策略和迁移架构是影响方法效果的关键因素。研究结果表明了知识迁移的可行性及其在增强LLMs各类软件工程任务性能方面的潜力。知识迁移的效果因目标领域和任务类型而异：分层策略比直接迁移更为有效，而在提示设计中AI-Chain优于CoT。这些发现的启示超越了软件工程任务范畴，表明知识迁移可提升LLMs处理任何自然语言任务中未知情况的能力。