Despite the superb performance in many tasks, large language models (LLMs) bear the risk of generating hallucination or even wrong answers when confronted with tasks that demand the accuracy of knowledge. The issue becomes even more noticeable when addressing logic queries that require multiple logic reasoning steps. On the other hand, knowledge graph (KG) based question answering methods are capable of accurately identifying the correct answers with the help of knowledge graph, yet its accuracy could quickly deteriorate when the knowledge graph itself is sparse and incomplete. It remains a critical challenge on how to integrate knowledge graph reasoning with LLMs in a mutually beneficial way so as to mitigate both the hallucination problem of LLMs as well as the incompleteness issue of knowledge graphs. In this paper, we propose 'Logic-Query-of-Thoughts' (LGOT) which is the first of its kind to combine LLMs with knowledge graph based logic query reasoning. LGOT seamlessly combines knowledge graph reasoning and LLMs, effectively breaking down complex logic queries into easy to answer subquestions. Through the utilization of both knowledge graph reasoning and LLMs, it successfully derives answers for each subquestion. By aggregating these results and selecting the highest quality candidate answers for each step, LGOT achieves accurate results to complex questions. Our experimental findings demonstrate substantial performance enhancements, with up to 20% improvement over ChatGPT.

翻译：尽管大语言模型（LLMs）在许多任务中表现出色，但在面对需要知识准确性的任务时，它们存在产生幻觉甚至错误答案的风险。当处理需要多步逻辑推理的逻辑查询时，这一问题尤为显著。另一方面，基于知识图谱（KG）的问答方法能够借助知识图谱准确识别正确答案，然而当知识图谱本身稀疏或不完整时，其准确性会迅速下降。如何以互惠互利的方式将知识图谱推理与大语言模型相结合，从而同时缓解LLMs的幻觉问题和知识图谱的不完整性问题，仍然是一个关键挑战。在本文中，我们提出了“逻辑查询思维”（LGOT），这是首个将LLMs与基于知识图谱的逻辑查询推理相结合的方法。LGOT无缝融合了知识图谱推理与LLMs，有效将复杂逻辑查询分解为易于回答的子问题。通过同时利用知识图谱推理和LLMs，它为每个子问题成功推导出答案。通过汇总这些结果并为每一步选择最高质量的候选答案，LGOT实现了对复杂问题的准确回答。我们的实验结果表明，与ChatGPT相比，性能显著提升达20%。