An emerging area of research in situated and multimodal interactive conversations (SIMMC) includes interactions in scientific papers. Since scientific papers are primarily composed of text, equations, figures, and tables, SIMMC methods must be developed specifically for each component to support the depth of inquiry and interactions required by research scientists. This work introduces Conversational Papers (cPAPERS), a dataset of conversational question-answer pairs from reviews of academic papers grounded in these paper components and their associated references from scientific documents available on arXiv. We present a data collection strategy to collect these question-answer pairs from OpenReview and associate them with contextual information from LaTeX source files. Additionally, we present a series of baseline approaches utilizing Large Language Models (LLMs) in both zero-shot and fine-tuned configurations to address the cPAPERS dataset.

翻译：情境化多模态交互对话（SIMMC）的新兴研究领域涵盖科学论文中的交互。由于科学论文主要由文本、公式、图表和表格构成，必须针对每个组成部分专门开发SIMMC方法，以支持科研人员所需的深度探究与交互。本研究提出对话式论文数据集（cPAPERS），该数据集包含基于arXiv科学文档中论文组成部分及其关联参考文献的学术论文评审对话问答对。我们提出一种数据收集策略，从OpenReview平台收集这些问答对，并将其与LaTeX源文件中的上下文信息进行关联。此外，我们提出一系列基于大语言模型（LLM）的基线方法，包括零样本配置与微调配置，以处理cPAPERS数据集。