Foundation models, such as Large language Models (LLMs), have attracted significant amount of interest due to their large number of applications. However, when handling tasks involving repetitive sub-tasks and/or deceptive contents, such as arithmetic calculation and article-level fake news detection, simple instructional prompts suffer from inaccurate responses. Existing works show that more complicated prompting strategies, such as Chain-of-Thoughts and Least-to-Most, can unlock LLM's powerful capacity in diverse areas. Recent researches reveal that simple divide-and-conquer prompting strategy, i.e. simply dividing the input sequence to multiple sub-inputs, can also substantially improve LLM's performance in some specific tasks such as misinformation detection. In this paper, we aim at examining the utility of divide-and-conquer prompting strategy and answer on which kind of tasks this strategy gets advantages. Specifically, we provide a theoretic analysis to divide-and-conquer prompting strategy and help us identify the specific tasks where DaC prompting can bring performance boost with theoretic guarantee. We then present two cases (large integer arithmetic and fact verification) where experimental results aligns with our theoretic analysis.
翻译:基础模型(如大语言模型)因其广泛的应用前景而受到极大关注。然而,在处理涉及重复子任务和/或欺骗性内容的任务时(例如算术运算与文章级虚假新闻检测),简单的指令提示往往导致响应失准。现有研究表明,更复杂的提示策略(如思维链和由简至繁提示法)能够释放大语言模型在多领域的强大能力。近期研究揭示,简单的分治提示策略——即仅将输入序列拆分为多个子输入——也能显著提升大语言模型在特定任务(如虚假信息检测)中的表现。本文旨在检验分治提示策略的效用,并回答该策略在何种任务中具有优势。具体而言,我们首先对分治提示策略进行理论分析,通过理论保证帮助识别分治提示能够带来性能提升的具体任务类型。随后通过两个案例(大整数运算与事实核查)展示实验结果与理论分析的一致性。