Breaking down a document or a conversation into multiple contiguous segments based on its semantic structure is an important and challenging problem in NLP, which can assist many downstream tasks. However, current works on topic segmentation often focus on segmentation of structured texts. In this paper, we comprehensively analyze the generalization capabilities of state-of-the-art topic segmentation models on unstructured texts. We find that: (a) Current strategies of pre-training on a large corpus of structured text such as Wiki-727K do not help in transferability to unstructured conversational data. (b) Training from scratch with only a relatively small-sized dataset of the target unstructured domain improves the segmentation results by a significant margin. We stress-test our proposed Topic Segmentation approach by experimenting with multiple loss functions, in order to mitigate effects of imbalance in unstructured conversational datasets. Our empirical evaluation indicates that Focal Loss function is a robust alternative to Cross-Entropy and re-weighted Cross-Entropy loss function when segmenting unstructured and semi-structured chats.

翻译：将文档或对话基于其语义结构分解为多个连续片段，是自然语言处理中重要且具有挑战性的问题，可辅助众多下游任务。然而，当前关于主题分割的研究通常聚焦于结构化文本的分割。本文全面分析了最先进的主题分割模型在非结构化文本上的泛化能力。我们发现：(a) 当前在Wiki-727K等大规模结构化文本语料上进行预训练的策略，并未提升其对非结构化对话数据的迁移能力；(b) 仅使用目标非结构化领域的较小数据集进行从头训练，可显著改善分割结果。我们通过实验多种损失函数对提出的主题分割方法进行压力测试，以缓解非结构化对话数据集中的类别不平衡影响。实证评估表明，在分割非结构化与半结构化聊天内容时，Focal Loss函数是比交叉熵损失函数和重新加权交叉熵损失函数更稳健的替代方案。