While large language models (LLMs) can answer many questions correctly, they can also hallucinate and give wrong answers. Wikidata, with its over 12 billion facts, can be used to ground LLMs to improve their factuality. This paper presents WikiWebQuestions, a high-quality question answering benchmark for Wikidata. Ported over from WebQuestions for Freebase, it consists of real-world data with SPARQL annotation. This paper presents a few-shot sequence-to-sequence semantic parser for Wikidata. We modify SPARQL to use the unique domain and property names instead of their IDs. We train the parser to use either the results from an entity linker or mentions in the query. We fine-tune LLaMA by adding the few-shot training data to that used to fine-tune Alpaca. Our experimental results demonstrate the effectiveness of this methodology, establishing a strong baseline of 76% and 65% answer accuracy in the dev and test sets of WikiWebQuestions, respectively. By pairing our semantic parser with GPT-3, we combine verifiable results with qualified GPT-3 guesses to provide useful answers to 96% of the questions in dev. We also show that our method outperforms the state-of-the-art for the QALD-7 Wikidata dataset by 3.6% in F1 score.

翻译：尽管大型语言模型（LLMs）能够正确回答许多问题，但它们也可能产生幻觉并给出错误答案。维基数据（Wikidata）拥有超过120亿条事实，可用于约束LLM以提升其事实准确性。本文提出WikiWebQuestions——一个面向维基数据的高质量问答基准测试集。该数据集从面向Freebase的WebQuestions迁移而来，包含带有SPARQL标注的真实世界数据。我们提出一种面向维基数据的少样本序列到序列语义解析方法，将SPARQL修改为使用唯一领域和属性名称而非其ID。我们训练解析器利用实体链接器的结果或查询中的提及信息。通过将少样本训练数据添加到用于微调Alpaca的数据集中，我们对LLaMA进行微调。实验结果表明了该方法的有效性：在WikiWebQuestions的开发集和测试集上分别建立了76%和65%答案准确率的强基线。通过将语义解析器与GPT-3结合，我们将可验证结果与GPT-3的合格猜测相结合，为开发集中96%的问题提供有用答案。我们还表明，我们的方法在QALD-7维基数据数据集上的F1分数比当前最优方法高出3.6%。