Despite the superb performance in many tasks, large language models (LLMs) bear the risk of generating hallucination or even wrong answers when confronted with tasks that demand the accuracy of knowledge. The issue becomes even more noticeable when addressing logic queries that require multiple logic reasoning steps. On the other hand, knowledge graph (KG) based question answering methods are capable of accurately identifying the correct answers with the help of knowledge graph, yet its accuracy could quickly deteriorate when the knowledge graph itself is sparse and incomplete. It remains a critical challenge on how to integrate knowledge graph reasoning with LLMs in a mutually beneficial way so as to mitigate both the hallucination problem of LLMs as well as the incompleteness issue of knowledge graphs. In this paper, we propose 'Logic-Query-of-Thoughts' (LGOT) which is the first of its kind to combine LLMs with knowledge graph based logic query reasoning. LGOT seamlessly combines knowledge graph reasoning and LLMs, effectively breaking down complex logic queries into easy to answer subquestions. Through the utilization of both knowledge graph reasoning and LLMs, it successfully derives answers for each subquestion. By aggregating these results and selecting the highest quality candidate answers for each step, LGOT achieves accurate results to complex questions. Our experimental findings demonstrate substantial performance enhancements, with up to 20% improvement over ChatGPT.

翻译：尽管在许多任务中表现出色，大型语言模型（LLMs）在面对需要知识准确性的任务时，仍存在产生幻觉甚至错误答案的风险。当处理需要多步逻辑推理的逻辑查询时，这一问题尤为显著。另一方面，基于知识图谱（KG）的问答方法能够借助知识图谱准确识别正确答案，但当知识图谱本身稀疏且不完整时，其准确性会迅速下降。如何以互利的方式将知识图谱推理与LLMs相结合，从而同时缓解LLMs的幻觉问题和知识图谱的不完整性问题，仍然是一个关键挑战。本文提出“逻辑查询思维”（LGOT）方法，首次将LLMs与基于知识图谱的逻辑查询推理相结合。LGOT无缝整合了知识图谱推理和LLMs，有效将复杂逻辑查询分解为易于回答的子问题。通过同时利用知识图谱推理和LLMs，该方法成功推导出每个子问题的答案。通过聚合这些结果并为每个步骤选择最高质量的候选答案，LGOT能够对复杂问题得出准确结果。我们的实验结果表明，该方法实现了显著的性能提升，较ChatGPT最高可提升20%。