Despite the superb performance in many tasks, large language models (LLMs) bear the risk of generating hallucination or even wrong answers when confronted with tasks that demand the accuracy of knowledge. The issue becomes even more noticeable when addressing logic queries that require multiple logic reasoning steps. On the other hand, knowledge graph (KG) based question answering methods are capable of accurately identifying the correct answers with the help of knowledge graph, yet its accuracy could quickly deteriorate when the knowledge graph itself is sparse and incomplete. It remains a critical challenge on how to integrate knowledge graph reasoning with LLMs in a mutually beneficial way so as to mitigate both the hallucination problem of LLMs as well as the incompleteness issue of knowledge graphs. In this paper, we propose 'Logic-Query-of-Thoughts' (LGOT) which is the first of its kind to combine LLMs with knowledge graph based logic query reasoning. LGOT seamlessly combines knowledge graph reasoning and LLMs, effectively breaking down complex logic queries into easy to answer subquestions. Through the utilization of both knowledge graph reasoning and LLMs, it successfully derives answers for each subquestion. By aggregating these results and selecting the highest quality candidate answers for each step, LGOT achieves accurate results to complex questions. Our experimental findings demonstrate substantial performance enhancements, with up to 20% improvement over ChatGPT.

翻译：尽管大型语言模型（LLMs）在许多任务中表现出色，但在处理需要知识准确性的任务时，仍存在产生幻觉甚至错误答案的风险。当涉及需要多步逻辑推理的逻辑查询时，这一问题尤为突出。另一方面，基于知识图谱（KG）的问答方法能够借助知识图谱准确识别正确答案，但当知识图谱本身稀疏且不完整时，其准确性会迅速下降。如何以互利共赢的方式将知识图谱推理与LLMs相结合，从而同时缓解LLMs的幻觉问题和知识图谱的不完整性，仍然是一个关键挑战。本文提出“逻辑思维查询”（LGOT），这是首次将LLMs与基于知识图谱的逻辑查询推理相结合的方法。LGOT无缝整合了知识图谱推理和LLMs，有效将复杂逻辑查询分解为易于回答的子问题。通过同时利用知识图谱推理和LLMs，它成功推导出每个子问题的答案。通过聚合这些结果并为每个步骤选择最高质量的候选答案，LGOT能够对复杂问题得出准确结果。我们的实验结果表明，该方法实现了显著的性能提升，较ChatGPT最高可提升20%。