Knowledge base question answering (KBQA) is a critical yet challenging task due to the vast number of entities within knowledge bases and the diversity of natural language questions posed by users. Unfortunately, the performance of most KBQA models tends to decline significantly in real-world scenarios where high-quality annotated data is insufficient. To mitigate the burden associated with manual annotation, we introduce FlexKBQA by utilizing Large Language Models (LLMs) as program translators for addressing the challenges inherent in the few-shot KBQA task. Specifically, FlexKBQA leverages automated algorithms to sample diverse programs, such as SPARQL queries, from the knowledge base, which are subsequently converted into natural language questions via LLMs. This synthetic dataset facilitates training a specialized lightweight model for the KB. Additionally, to reduce the barriers of distribution shift between synthetic data and real user questions, FlexKBQA introduces an executionguided self-training method to iterative leverage unlabeled user questions. Furthermore, we explore harnessing the inherent reasoning capability of LLMs to enhance the entire framework. Consequently, FlexKBQA delivers substantial flexibility, encompassing data annotation, deployment, and being domain agnostic. Through extensive experiments on GrailQA, WebQSP, and KQA Pro, we observe that under the few-shot even the more challenging zero-shot scenarios, FlexKBQA achieves impressive results with a few annotations, surpassing all previous baselines and even approaching the performance of supervised models, achieving a remarkable 93% performance relative to the fully-supervised models. We posit that FlexKBQA represents a significant advancement towards exploring better integration of large and lightweight models. The code is open-sourced.

翻译：知识库问答（KBQA）是一项关键且具有挑战性的任务，其原因在于知识库中实体数量庞大且用户提出的自然语言问题具有多样性。不幸的是，在高质量标注数据不足的实际场景中，大多数KBQA模型的性能往往会显著下降。为减轻手动标注的负担，我们通过利用大语言模型（LLMs）作为程序翻译器来应对少样本KBQA任务中的挑战，提出了FlexKBQA。具体而言，FlexKBQA借助自动化算法从知识库中采样多样化的程序（例如SPARQL查询），随后通过大语言模型将其转换为自然语言问题。这一合成数据集有助于为知识库训练一个专门的轻量级模型。此外，为降低合成数据与真实用户问题之间分布偏移的障碍，FlexKBQA引入了一种执行引导的自训练方法，以迭代方式利用未标注的用户问题。更进一步，我们探索了利用大语言模型固有的推理能力来增强整体框架。因此，FlexKBQA在数据标注、部署以及领域无关性方面均提供了显著的灵活性。通过在GrailQA、WebQSP和KQA Pro上的大量实验，我们观察到在少样本甚至更具挑战性的零样本场景下，FlexKBQA仅凭少量标注即可取得令人印象深刻的结果，超越了所有先前的基线方法，甚至接近了监督模型的性能，达到了全监督模型93%的显著性能。我们认为FlexKBQA是朝着探索大模型与轻量级模型更好集成方向迈出的重要一步。相关代码已开源。