This paper aims to quantitatively evaluate the performance of ChatGPT, an interactive large language model, on inter-sentential relations such as temporal relations, causal relations, and discourse relations. Given ChatGPT's promising performance across various tasks, we conduct extensive evaluations on the whole test sets of 13 datasets, including temporal and causal relations, PDTB2.0-based and dialogue-based discourse relations, and downstream applications on discourse understanding. To achieve reliable results, we adopt three tailored prompt templates for each task, including the zero-shot prompt template, zero-shot prompt engineering (PE) template, and in-context learning (ICL) prompt template, to establish the initial baseline scores for all popular sentence-pair relation classification tasks for the first time. We find that ChatGPT exhibits strong performance in detecting and reasoning about causal relations, while it may not be proficient in identifying the temporal order between two events. It can recognize most discourse relations with existing explicit discourse connectives, but the implicit discourse relation still remains a challenging task. Meanwhile, ChatGPT performs poorly in the dialogue discourse parsing task that requires structural understanding in a dialogue before being aware of the discourse relation.

翻译：本文旨在定量评估交互式大语言模型ChatGPT在句间关系（包括时间关系、因果关系和话语关系）中的表现。鉴于ChatGPT在多项任务中展现的优异性能，我们在13个数据集的全部测试集上进行了广泛评估，涵盖时间与因果关系、基于PDTB2.0和对话的话语关系，以及下游话语理解应用。为获得可靠结果，我们针对每项任务设计了三类定制化提示模板：零样本提示模板、零样本提示工程模板和上下文学习提示模板，首次为所有主流通用的句子对关系分类任务建立了初始基线评分。实验发现：ChatGPT在因果关系检测与推理方面表现强劲，但在识别两个事件的时间顺序上存在不足；它能借助显式话语连接词识别大多数话语关系，但隐式话语关系仍是挑战性任务；同时，在需要先理解对话结构再判断话语关系的对话话语解析任务中，ChatGPT表现欠佳。