Large language models (LLMs) have demonstrated remarkable performance by following natural language instructions without fine-tuning them on domain-specific tasks and data. However, leveraging LLMs for domain-specific question answering suffers from severe limitations. The generated answer tends to hallucinate due to the training data collection time (when using off-the-shelf), complex user utterance and wrong retrieval (in retrieval-augmented generation). Furthermore, due to the lack of awareness about the domain and expected output, such LLMs may generate unexpected and unsafe answers that are not tailored to the target domain. In this paper, we propose CarExpert, an in-car retrieval-augmented conversational question-answering system leveraging LLMs for different tasks. Specifically, CarExpert employs LLMs to control the input, provide domain-specific documents to the extractive and generative answering components, and controls the output to ensure safe and domain-specific answers. A comprehensive empirical evaluation exhibits that CarExpert outperforms state-of-the-art LLMs in generating natural, safe and car-specific answers.

翻译：大语言模型（LLMs）通过遵循自然语言指令，无需针对特定领域任务和数据微调便展现出卓越性能。然而，将LLMs应用于特定领域的问答时存在严重局限性：由于训练数据收集时间（使用现成模型时）、用户复杂表述以及检索增强生成中的错误检索，生成的答案容易产生幻觉。此外，由于缺乏对目标领域及预期输出的认知，此类LLMs可能生成不符合领域特性的意外甚至危险答案。本文提出CarExpert——一种基于检索增强的车载对话式问答系统，利用LLMs完成多类任务。具体而言，CarExpert通过LLMs控制输入、为抽取式及生成式问答组件提供领域专属文档，并管控输出以确保生成安全且符合领域要求的答案。全面的实证评估表明，CarExpert在生成自然、安全且具汽车特性的答案方面优于现有最先进的LLMs。