Despite the success of large language models (LLMs) in various natural language processing (NLP) tasks, the stored knowledge in these models may inevitably be incomplete, out-of-date, or incorrect. This motivates the need to utilize external knowledge to assist LLMs. Unfortunately, current methods for incorporating external knowledge often require additional training or fine-tuning, which can be costly and may not be feasible for LLMs. To address this issue, we propose a novel post-processing approach, rethinking with retrieval (RR), which retrieves relevant external knowledge based on the decomposed reasoning steps obtained from the chain-of-thought (CoT) prompting. This lightweight approach does not require additional training or fine-tuning and is not limited by the input length of LLMs. We evaluate the effectiveness of RR through extensive experiments with GPT-3 on three complex reasoning tasks: commonsense reasoning, temporal reasoning, and tabular reasoning. Our results show that RR can produce more faithful explanations and improve the performance of LLMs.

翻译：尽管大型语言模型(LLMs)在各种自然语言处理(NLP)任务中取得了成功,但这些模型中储存的知识可能不可避免地不完全、过时或不正确,这促使需要利用外部知识协助LLMs。不幸的是,目前纳入外部知识的方法往往需要额外的培训或微调,这对于LLMs来说可能成本很高,而且可能不可行。为了解决这个问题,我们提出一种新的后处理方法,与检索(RR)一起重新思考,根据从思维链(CoT)中激发的分解的推理步骤检索相关的外部知识。这种轻便方法不需要额外的培训或微调,而且不受LLMs投入长度的限制。我们通过与GPT-3就三种复杂的推理任务进行广泛的实验来评估R的有效性:常识推理、时间推理和图表推理。我们的结果表明,RR可以产生更忠实的解释,提高LMs的性能。