Interaction with Large Language Models (LLMs) is primarily carried out via prompting. A prompt is a natural language instruction designed to elicit certain behaviour or output from a model. In theory, natural language prompts enable non-experts to interact with and leverage LLMs. However, for complex tasks and tasks with specific requirements, prompt design is not trivial. Creating effective prompts requires skill and knowledge, as well as significant iteration in order to determine model behavior, and guide the model to accomplish a particular goal. We hypothesize that the way in which users iterate on their prompts can provide insight into how they think prompting and models work, as well as the kinds of support needed for more efficient prompt engineering. To better understand prompt engineering practices, we analyzed sessions of prompt editing behavior, categorizing the parts of prompts users iterated on and the types of changes they made. We discuss design implications and future directions based on these prompt engineering practices.

翻译：与大型语言模型（LLMs）的交互主要通过提示（prompt）实现。提示是一种旨在引导模型产生特定行为或输出的自然语言指令。理论上，自然语言提示可使非专家用户与LLMs进行交互并加以利用。然而，针对复杂任务及具有特殊要求的任务而言，提示设计并非易事。创建有效提示需具备技能和知识，同时需要通过反复迭代来明确模型行为，进而引导模型达成特定目标。我们假设：用户迭代提示的方式能揭示其对提示机制和模型运作的认知，以及高效提示工程所需的支持类型。为了更深入地理解提示工程实践，我们对用户编辑提示的行为会话进行了分析，对用户迭代优化的提示组成部分及其变更类型进行了分类。基于这些提示工程实践，我们探讨了设计启示与未来研究方向。