Large language models (LLMs) can solve problems step-by-step. While this chain-of-thought (CoT) reasoning boosts LLMs' performance, it is unclear if LLMs \textit{know} when to use CoT and whether those CoT are always necessary to answer the question. This paper shows that LLMs tend to generate redundant calculations and reasoning on a manually constructed math QA dataset, GSM8K-Zero. GSM8K-Zero is constructed such that the questions can be answered without any calculations, but LLMs, including Llama-2 models and Claude-2, tend to generate lengthy and unnecessary calculations to answer the questions. We also conduct experiments to explain why LLMs generate redundant calculations and reasonings. GSM8K-Zero is publicly available at https://github.com/d223302/Over-Reasoning-of-LLMs and https://huggingface.co/datasets/dcml0714/GSM8K-Zero.

翻译：大型语言模型能够逐步解决问题。虽然这种思维链推理提升了模型的性能，但尚不清楚模型是否明确知道何时该使用思维链，以及这些思维链对于回答问题是否总是必要的。本文表明，在人工构建的数学问答数据集GSM8K-Zero上，大型语言模型倾向于生成冗余计算和推理。GSM8K-Zero被设计为无需任何计算即可回答问题的问题集，但包括Llama-2系列模型和Claude-2在内的大型语言模型，仍会生成冗长且不必要的计算来回答问题。我们还通过实验解释了大型语言模型产生冗余计算和推理的原因。GSM8K-Zero已在https://github.com/d223302/Over-Reasoning-of-LLMs和https://huggingface.co/datasets/dcml0714/GSM8K-Zero公开提供。