Knowledge base question answering (KBQA) is a critical yet challenging task due to the vast number of entities within knowledge bases and the diversity of natural language questions posed by users. Unfortunately, the performance of most KBQA models tends to decline significantly in real-world scenarios where high-quality annotated data is insufficient. To mitigate the burden associated with manual annotation, we introduce FlexKBQA by utilizing Large Language Models (LLMs) as program translators for addressing the challenges inherent in the few-shot KBQA task. Specifically, FlexKBQA leverages automated algorithms to sample diverse programs, such as SPARQL queries, from the knowledge base, which are subsequently converted into natural language questions via LLMs. This synthetic dataset facilitates training a specialized lightweight model for the KB. Additionally, to reduce the barriers of distribution shift between synthetic data and real user questions, FlexKBQA introduces an executionguided self-training method to iterative leverage unlabeled user questions. Furthermore, we explore harnessing the inherent reasoning capability of LLMs to enhance the entire framework. Consequently, FlexKBQA delivers substantial flexibility, encompassing data annotation, deployment, and being domain agnostic. Through extensive experiments on GrailQA, WebQSP, and KQA Pro, we observe that under the few-shot even the more challenging zero-shot scenarios, FlexKBQA achieves impressive results with a few annotations, surpassing all previous baselines and even approaching the performance of supervised models, achieving a remarkable 93% performance relative to the fully-supervised models. We posit that FlexKBQA represents a significant advancement towards exploring better integration of large and lightweight models. The code is open-sourced.

翻译：知识库问答是至关重要的挑战性任务，其难点在于知识库中实体数量庞大且用户自然语言问题形式多样。然而，在高质量标注数据不足的真实场景中，大多数知识库问答模型的性能往往显著下降。为减轻人工标注负担，我们提出FlexKBQA，利用大型语言模型作为程序翻译器应对少样本知识库问答任务的固有挑战。具体而言，FlexKBQA通过自动化算法从知识库中采样多样化程序（如SPARQL查询），再经大型语言模型将其转换为自然语言问题。该合成数据集有助于训练针对特定知识库的轻量级专用模型。为降低合成数据与真实用户问题之间的分布偏移障碍，FlexKBQA引入执行引导自训练方法迭代利用未标注用户问题。此外，我们探索利用大型语言模型的固有推理能力增强整体框架。最终FlexKBQA在数据标注、模型部署及跨领域应用方面均展现出显著灵活性。在GrailQA、WebQSP和KQA Pro上的大量实验表明，在少样本乃至更具挑战性的零样本场景下，FlexKBQA仅需少量标注即可取得令人瞩目的结果，超越所有先前基线方法，其性能甚至逼近全监督模型——达到全监督模型93%的优异表现。我们认为FlexKBQA在探索大型模型与轻量级模型更好融合方面实现了重要突破。相关代码已开源。