Prompting LLMs for complex tasks (e.g., building a trip advisor chatbot) needs humans to clearly articulate customized requirements (e.g., "start the response with a tl;dr"). However, existing prompt engineering instructions often lack focused training on requirement articulation and instead tend to emphasize increasingly automatable strategies (e.g., tricks like adding role-plays and "think step-by-step"). To address the gap, we introduce Requirement-Oriented Prompt Engineering (ROPE), a paradigm that focuses human attention on generating clear, complete requirements during prompting. We implement ROPE through an assessment and training suite that provides deliberate practice with LLM-generated feedback. In a randomized controlled experiment with 30 novices, ROPE significantly outperforms conventional prompt engineering training (20% vs. 1% gains), a gap that automatic prompt optimization cannot close. Furthermore, we demonstrate a direct correlation between the quality of input requirements and LLM outputs. Our work paves the way to empower more end-users to build complex LLM applications.
翻译:为复杂任务(例如构建旅行顾问聊天机器人)提示大语言模型需要人类清晰地阐明定制化需求(例如“以摘要开始回答”)。然而,现有的提示工程指导往往缺乏对需求阐述的重点培训,反而倾向于强调日益可自动化的策略(例如添加角色扮演和“逐步思考”等技巧)。为弥补这一差距,我们引入了需求导向的提示工程,这是一种将人类注意力集中在提示过程中生成清晰、完整需求的范式。我们通过一个评估与培训套件来实现ROPE,该套件提供带有大语言模型生成反馈的刻意练习。在一项涉及30名新手的随机对照实验中,ROPE显著优于传统的提示工程培训(提升20%对比1%),这一差距是自动提示优化无法弥合的。此外,我们证明了输入需求的质量与大语言模型输出之间存在直接相关性。我们的工作为赋能更多终端用户构建复杂的大语言模型应用铺平了道路。