This paper is on the problem of Knowledge-Based Visual Question Answering (KB-VQA). Recent works have emphasized the significance of incorporating both explicit (through external databases) and implicit (through LLMs) knowledge to answer questions requiring external knowledge effectively. A common limitation of such approaches is that they consist of relatively complicated pipelines and often heavily rely on accessing GPT-3 API. Our main contribution in this paper is to propose a much simpler and readily reproducible pipeline which, in a nutshell, is based on efficient in-context learning by prompting LLaMA (1 and 2) using question-informative captions as contextual information. Contrary to recent approaches, our method is training-free, does not require access to external databases or APIs, and yet achieves state-of-the-art accuracy on the OK-VQA and A-OK-VQA datasets. Finally, we perform several ablation studies to understand important aspects of our method. Our code is publicly available at https://github.com/alexandrosXe/ASimple-Baseline-For-Knowledge-Based-VQA

翻译：本文针对知识驱动的视觉问答（KB-VQA）问题展开研究。近期工作强调了在回答需要外部知识的问题时，有效整合显性知识（通过外部数据库）与隐性知识（通过大型语言模型）的重要性。此类方法普遍存在管道复杂、且常严重依赖GPT-3 API调用的局限性。本文的核心贡献在于提出一种更简洁且易于复现的流水线方案，其核心思想是通过将包含问题信息的图像描述作为上下文信息，以高效上下文学习方式提示LLaMA（1代和2代）模型。与近期方法相反，我们的方法无需训练，无需访问外部数据库或API，却在OK-VQA和A-OK-VQA数据集上达到了最先进的准确率。最后，我们通过多项消融实验深入剖析了方法的关键要素。相关代码已开源至 https://github.com/alexandrosXe/ASimple-Baseline-For-Knowledge-Based-VQA