Large Language Models (LLMs) perform well on basic programming problems. However, they encounter challenges when dealing with complex tasks involving the use of diverse algorithmic and data structure skills, particularly programming competition-level problems. Notably, ChatGPT exhibits proficient performance on problems it has encountered during its pre-training phase, but this performance deteriorates when faced with novel problems. Consequently, enhancing the ability of LLMs to address unfamiliar problems has emerged as a pivotal research focus. The problem-solving process of LLMs mirrors human programmers' approach to a certain extent. When confronted with new programming tasks, human programmers engage in task planning and code writing with the previously acquired knowledge about algorithms and data structures. Despite having learned such knowledge, LLMs struggle to effectively apply it when faced with specific new problems. To address this issue, we constructed a novel dataset, CodeF, which contains a portion of programming problems that ChatGPT has not previously encountered. Furthermore, we developed a Knowledge Library tailored for Python programming contest problems and introduced the concept of Knowledge-Aware Code Generation (KareCoder). KareCoder bolsters the models' understanding and problem-solving capabilities by integrating prompt and knowledge from the library into the LLMs' code generation reasoning process, especially on Pass@1 metrics. Upon testing on the CodeF and APPS datasets, KareCoder demonstrated outstanding performance in handling novel problems previously unencountered by LLMs. In contrast with the code directly generated by ChatGPT, KareCoder achieved a relative improvement of 23.3% on the Pass@1 metric on the CodeF post2021-9 dataset. Additionally, it performs well compared to other methods when dealing with problems that LLMs have previously encountered.

翻译：大型语言模型（LLMs）在基础编程问题上表现良好。然而，在处理涉及多种算法和数据结构技能的复杂任务（特别是编程竞赛级别的问题）时，它们遇到了挑战。值得注意的是，ChatGPT 在其预训练阶段遇到过的问题上表现出熟练的性能，但在面对新问题时性能下降。因此，提升 LLMs 解决陌生问题的能力已成为关键研究方向。LLMs 的解题过程在一定程度上模仿了人类程序员的方法。当面对新的编程任务时，人类程序员会利用先前获得的算法和数据结构知识进行任务规划和代码编写。尽管 LLMs 已经学习了这些知识，但它们在具体新问题面前难以有效应用。为解决此问题，我们构建了一个新数据集 CodeF，其中包含一部分 ChatGPT 之前未遇到的编程问题。此外，我们开发了一个针对 Python 编程竞赛问题的知识库，并引入了知识感知代码生成（KareCoder）的概念。KareCoder 通过将提示和知识库中的知识整合到 LLMs 的代码生成推理过程中，增强了模型的理解和解题能力，尤其在 Pass@1 指标上。在 CodeF 和 APPS 数据集上的测试中，KareCoder 在处理 LLMs 之前未遇到的新问题时表现出色。与 ChatGPT 直接生成的代码相比，KareCoder 在 CodeF post2021-9 数据集的 Pass@1 指标上实现了 23.3% 的相对提升。同时，在处理 LLMs 之前遇到过的问题时，它与其他方法相比表现良好。