Prompt design plays a crucial role in shaping the efficacy of ChatGPT, influencing the model's ability to extract contextually accurate responses. Thus, optimal prompt construction is essential for maximizing the utility and performance of ChatGPT. However, sub-optimal prompt design may necessitate iterative refinement, as imprecise or ambiguous instructions can lead to undesired responses from ChatGPT. Existing studies explore several prompt patterns and strategies to improve the relevance of responses generated by ChatGPT. However, the exploration of constraints that necessitate the submission of multiple prompts is still an unmet attempt. In this study, our contributions are twofold. First, we attempt to uncover gaps in prompt design that demand multiple iterations. In particular, we manually analyze 686 prompts that were submitted to resolve issues related to Java and Python programming languages and identify eleven prompt design gaps (e.g., missing specifications). Such gap exploration can enhance the efficacy of single prompts in ChatGPT. Second, we attempt to reproduce the ChatGPT response by consolidating multiple prompts into a single one. We can completely consolidate prompts with four gaps (e.g., missing context) and partially consolidate prompts with three gaps (e.g., additional functionality). Such an effort provides concrete evidence to users to design more optimal prompts mitigating these gaps. Our study findings and evidence can - (a) save users time, (b) reduce costs, and (c) increase user satisfaction.
翻译:提示设计在塑造ChatGPT效能中起着关键作用,影响着模型提取上下文准确响应的能力。因此,最优提示构建对于最大化ChatGPT的效用和性能至关重要。然而,次优的提示设计可能需要迭代优化,因为不精确或模糊的指令会导致ChatGPT生成不理想的响应。现有研究探索了多种提示模式与策略以提升ChatGPT生成结果的相关性,但针对需要提交多个提示的约束条件的探索仍是一个尚未满足的研究方向。本研究具有双重贡献:首先,我们试图揭示需要多轮迭代的提示设计缺陷。具体而言,我们人工分析了686个用于解决Java和Python编程语言相关问题的提示,识别出十一类提示设计缺陷(例如缺失规格说明)。这种缺陷探索可提升ChatGPT中单个提示的效能。其次,我们尝试通过将多个提示整合为单一提示来复现ChatGPT响应。我们可完全整合具有四类缺陷(如缺失上下文)的提示,并部分整合具有三类缺陷(例如额外功能)的提示。这项努力为用户设计更优提示以规避这些缺陷提供了具体依据。本研究的发现与证据能够:(a)节省用户时间,(b)降低使用成本,(c)提升用户满意度。