Since the launch of ChatGPT, a powerful AI Chatbot developed by OpenAI, large language models (LLMs) have made significant advancements in both academia and industry, bringing about a fundamental engineering paradigm shift in many areas. While LLMs are powerful, it is also crucial to best use their power where "prompt'' plays a core role. However, the booming LLMs themselves, including excellent APIs like ChatGPT, have several inherent limitations: 1) temporal lag of training data, and 2) the lack of physical capabilities to perform external actions. Recently, we have observed the trend of utilizing prompt-based tools to better utilize the power of LLMs for downstream tasks, but a lack of systematic literature and standardized terminology, partly due to the rapid evolution of this field. Therefore, in this work, we survey related prompting tools and promote the concept of the "Prompting Framework" (PF), i.e. the framework for managing, simplifying, and facilitating interaction with large language models. We define the lifecycle of the PF as a hierarchical structure, from bottom to top, namely: Data Level, Base Level, Execute Level, and Service Level. We also systematically depict the overall landscape of the emerging PF field and discuss potential future research and challenges. To continuously track the developments in this area, we maintain a repository at https://github.com/lxx0628/Prompting-Framework-Survey, which can be a useful resource sharing platform for both academic and industry in this field.

翻译：自OpenAI开发的强大AI聊天机器人ChatGPT发布以来，大语言模型在学术界和工业界均取得了显著进展，并在众多领域引发了根本性的工程范式转变。尽管大语言模型功能强大，但如何最优地利用其能力至关重要，其中"提示"扮演着核心角色。然而，蓬勃发展的大语言模型本身，包括ChatGPT等优秀API，存在几个固有局限性：1）训练数据的时间滞后性，以及2）缺乏执行外部操作的物理能力。近期，我们观察到利用基于提示的工具来更好地发挥大语言模型在下游任务中能力的趋势，但由于该领域的快速演进，相关系统文献和标准化术语仍较为匮乏。因此，本研究对相关提示工具进行综述，并推广"提示框架"这一概念，即用于管理、简化和促进与大语言模型交互的框架。我们将提示框架的生命周期定义为自底向上的层次结构：数据层、基础层、执行层和服务层。同时，我们系统描绘了新兴提示框架领域的整体图景，并探讨了潜在的未来研究方向与挑战。为持续追踪该领域发展动态，我们维护了GitHub仓库（https://github.com/lxx0628/Prompting-Framework-Survey），旨在为学术界和工业界提供实用的资源共享平台。