Prompting methods play a crucial role in enhancing the capabilities of pre-trained large language models (LLMs). We explore how contrastive prompting (CP) significantly improves the ability of large language models to perform complex reasoning. We demonstrate that LLMs are decent contrastive reasoners by simply adding "Let's give a correct and a wrong answer." before LLMs provide answers. Experiments on two large language models show that zero-shot contrastive prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks without any hand-crafted few-shot examples, such as increasing the accuracy on GSM8K from 35.9% to 88.8% and AQUA-RAT from 41.3% to 62.2% with the state-of-the-art GPT-4 model. Our method not only surpasses zero-shot CoT and few-shot CoT in most arithmetic and commonsense reasoning tasks but also can seamlessly integrate with existing prompting methods, resulting in improved or comparable results when compared to state-of-the-art methods. Our code is available at https://github.com/yao8839836/cp
翻译:提示方法在增强预训练大语言模型(LLMs)能力方面发挥着关键作用。我们探索了对比提示(CP)如何显著提升大语言模型执行复杂推理的能力。通过在大语言模型提供答案前简单添加"让我们给出一个正确答案和一个错误答案",我们证明了LLMs是优秀的对比推理器。在两个大语言模型上的实验表明,无需任何人工设计的少样本示例,零样本对比提示即可在算术、常识和符号推理任务上提升性能,例如使用最先进的GPT-4模型将GSM8K的准确率从35.9%提升至88.8%,AQUA-RAT从41.3%提升至62.2%。我们的方法不仅在大多数算术和常识推理任务中超越了零样本CoT和少样本CoT,还能与现有提示方法无缝集成,相比最先进方法取得更优或相当的结果。我们的代码已发布于https://github.com/yao8839836/cp。