Large language models, like ChatGPT, have shown remarkable capability in many downstream tasks, yet their ability to understand discourse structures of dialogues remains less explored, where it requires higher level capabilities of understanding and reasoning. In this paper, we aim to systematically inspect ChatGPT's performance in two discourse analysis tasks: topic segmentation and discourse parsing, focusing on its deep semantic understanding of linear and hierarchical discourse structures underlying dialogue. To instruct ChatGPT to complete these tasks, we initially craft a prompt template consisting of the task description, output format, and structured input. Then, we conduct experiments on four popular topic segmentation datasets and two discourse parsing datasets. The experimental results showcase that ChatGPT demonstrates proficiency in identifying topic structures in general-domain conversations yet struggles considerably in specific-domain conversations. We also found that ChatGPT hardly understands rhetorical structures that are more complex than topic structures. Our deeper investigation indicates that ChatGPT can give more reasonable topic structures than human annotations but only linearly parses the hierarchical rhetorical structures. In addition, we delve into the impact of in-context learning (e.g., chain-of-thought) on ChatGPT and conduct the ablation study on various prompt components, which can provide a research foundation for future work. The code is available at \url{https://github.com/yxfanSuda/GPTforDDA}.

翻译：大型语言模型（如ChatGPT）在诸多下游任务中展现出卓越能力，但其理解对话篇章结构的能力仍鲜有探究——这需要更高层次的推理与理解能力。本文旨在系统考察ChatGPT在两项篇章分析任务（话题分割与篇章解析）中的表现，重点关注其对对话中线性与层级篇章结构的深层语义理解。为引导ChatGPT完成这些任务，我们首先设计了包含任务描述、输出格式及结构化输入的提示模板，随后在四个主流话题分割数据集和两个篇章解析数据集上开展实验。结果表明：ChatGPT在通用领域对话中能有效识别话题结构，但在特定领域对话中表现欠佳；此外，我们发现ChatGPT难以理解比话题结构更复杂的修辞结构。深入研究发现，ChatGPT生成的话题结构较人工标注更合理，但对层级修辞结构仅能进行线性解析。我们进一步探究了上下文学习（如思维链）对ChatGPT的影响，并通过消融实验分析不同提示组件的效用，为后续研究奠定基础。实验代码已开源至\url{https://github.com/yxfanSuda/GPTforDDA}。