Large language models (LLMs) have exhibited remarkable ability in textual generation. However, in complex reasoning tasks such as code generation, generating the correct answer in a single attempt remains a formidable challenge for LLMs. Previous research has explored solutions by aggregating multiple outputs, leveraging the consistency among them. However, none of them have comprehensively captured this consistency from different perspectives. In this paper, we propose the Multi-Perspective Self-Consistency (MPSC) framework, a novel decoding strategy for LLM that incorporates both inter-consistency across outputs from multiple perspectives and intra-consistency within a single perspective. Specifically, we ask LLMs to sample multiple diverse outputs from various perspectives for a given query and then construct a multipartite graph based on them. With two predefined measures of consistency, we embed both inter- and intra-consistency information into the graph. The optimal choice is then determined based on consistency analysis in the graph. We conduct comprehensive evaluation on the code generation task by introducing solution, specification and test case as three perspectives. We leverage a code interpreter to quantitatively measure the inter-consistency and propose several intra-consistency measure functions. Our MPSC framework significantly boosts the performance on various popular benchmarks, including HumanEval (+17.60%), HumanEval Plus (+17.61%), MBPP (+6.50%) and CodeContests (+11.82%) in Pass@1, when compared to original outputs generated from ChatGPT, and even surpassing GPT-4.

翻译：大型语言模型在文本生成领域展现出卓越能力。然而在代码生成等复杂推理任务中，单次生成正确答案对这类模型仍是巨大挑战。已有研究通过聚合多个输出并利用输出间的一致性探索解决方案，但尚未有方法能从不同视角全面捕捉这种一致性。本文提出多视角自一致性（MPSC）框架，这是一种新型解码策略，同时融合了跨多个视角的输出间一致性与单视角内部的输出内一致性。具体而言，我们引导语言模型针对给定查询从多个不同视角生成多样化输出，并基于这些输出构建多部图。通过预定义两种一致性度量标准，我们将输出间一致性与输出内一致性信息嵌入图中，最终基于图的一致性分析确定最优选择。我们以解决方案、规约和测试用例作为三个视角，对代码生成任务进行了全面评估。利用代码解释器定量衡量输出间一致性，并提出了多种输出内一致性度量函数。与ChatGPT原始输出相比，我们的MPSC框架在多个流行基准测试中显著提升了Pass@1指标：HumanEval提升17.60%，HumanEval Plus提升17.61%，MBPP提升6.50%，CodeContests提升11.82%，甚至超越GPT-4的性能。