This paper aims to quantitatively evaluate the performance of ChatGPT, an interactive large language model, on inter-sentential relations such as temporal relations, causal relations, and discourse relations. Given ChatGPT's promising performance across various tasks, we conduct extensive evaluations on the whole test sets of 13 datasets, including temporal and causal relations, PDTB2.0-based and dialogue-based discourse relations, and downstream applications on discourse understanding. To achieve reliable results, we adopt three tailored prompt templates for each task, including the zero-shot prompt template, zero-shot prompt engineering (PE) template, and in-context learning (ICL) prompt template, to establish the initial baseline scores for all popular sentence-pair relation classification tasks for the first time. We find that ChatGPT exhibits strong performance in detecting and reasoning about causal relations, while it may not be proficient in identifying the temporal order between two events. It can recognize most discourse relations with existing explicit discourse connectives, but the implicit discourse relation still remains a challenging task. Meanwhile, ChatGPT performs poorly in the dialogue discourse parsing task that requires structural understanding in a dialogue before being aware of the discourse relation.

翻译：本文旨在定量评估交互式大语言模型ChatGPT在句间关系（如时序关系、因果关系和话语关系）上的性能。鉴于ChatGPT在多种任务中展现出的良好表现，我们对13个数据集的全部测试集进行了广泛评估，涵盖时序与因果关系、基于PDTB2.0及对话的话语关系，以及话语理解的下游应用。为确保结果可靠，我们为每项任务采用了三种定制提示模板：零样本提示模板、零样本提示工程模板和上下文学习提示模板，首次为所有主流句子对关系分类任务建立了初始基线得分。我们发现：ChatGPT在检测和推理因果关系方面表现强劲，但在识别两个事件之间的时序顺序上可能不够熟练；它能识别大多数带有显式话语连接词的话语关系，但隐式话语关系仍具挑战性；同时，在需要先理解对话结构才能感知话语关系的对话话语解析任务中，ChatGPT表现欠佳。