Automating data generation with Large Language Models (LLMs) has become increasingly popular. In this work, we investigate the feasibility and effectiveness of LLM-based data generation in the challenging setting of source-grounded information-seeking dialogs, with response attribution, over long documents. Our source texts consist of long and noisy meeting transcripts, adding to the task complexity. Since automating attribution remains difficult, we propose a semi-automatic approach: dialog queries and responses are generated with LLMs, followed by human verification and identification of attribution spans. Using this approach, we created MISeD -- Meeting Information Seeking Dialogs dataset -- a dataset of information-seeking dialogs focused on meeting transcripts. Models finetuned with MISeD demonstrate superior performance compared to off-the-shelf models, even those of larger size. Finetuning on MISeD gives comparable response generation quality to finetuning on fully manual data, while improving attribution quality and reducing time and effort.

翻译：利用大型语言模型（LLM）实现数据生成自动化已日益普及。本研究探讨了在具有挑战性的长文档源文本信息寻求对话场景中，基于LLM的数据生成的可行性与有效性，并实现了响应溯源。我们的源文本由冗长且含有噪声的会议转录本构成，这进一步增加了任务的复杂性。由于完全自动化溯源仍存在困难，我们提出了一种半自动化方法：首先使用LLM生成对话查询与响应，随后通过人工核查并标注溯源文本片段。基于该方法，我们构建了MISeD（会议信息寻求对话数据集）——一个专注于会议转录本的信息寻求对话数据集。使用MISeD微调的模型相较于现成模型（包括规模更大的模型）展现出更优的性能。在MISeD上的微调在响应生成质量方面与全人工数据微调相当，同时提升了溯源质量并显著减少了时间与人力成本。