Cross-lingual summarization (CLS) has attracted increasing interest in recent years due to the availability of large-scale web-mined datasets and the advancements of multilingual language models. However, given the rareness of naturally occurring CLS resources, the majority of datasets are forced to rely on translation which can contain overly literal artifacts. This restricts our ability to observe naturally occurring CLS pairs that capture organic diction, including instances of code-switching. This alteration between languages in mid-message is a common phenomenon in multilingual settings yet has been largely overlooked in cross-lingual contexts due to data scarcity. To address this gap, we introduce CroCoSum, a dataset of cross-lingual code-switched summarization of technology news. It consists of over 24,000 English source articles and 18,000 human-curated Chinese news summaries, with more than 92% of the summaries containing code-switched phrases. For reference, we evaluate the performance of existing approaches including pipeline, end-to-end, and zero-shot methods. We show that leveraging existing resources as a pretraining step does not improve performance on CroCoSum, indicating the limited generalizability of existing resources. Finally, we discuss the challenges of evaluating cross-lingual summarizers on code-switched generation through qualitative error analyses. Our collection and code can be accessed at https://github.com/RosenZhang/CroCoSum.

翻译：跨语言摘要近年来因大规模网络挖掘数据集和多语言语言模型的进步而受到广泛关注。然而，由于自然出现的跨语言摘要资源稀缺，多数数据集不得不依赖翻译，这可能导致过度直译的伪影。这限制了我们观察包含自然措辞（包括代码转换实例）的有机跨语言摘要对的能力。这种在对话过程中切换语言的现象在多语言环境中常见，但因数据匮乏而在跨语言情境中长期被忽视。为填补这一空白，我们提出CroCoSum——一个面向科技新闻的跨语言代码转换摘要数据集。该数据集包含超过24,000篇英文源文章和18,000条人工整理的新闻中文摘要，其中超过92%的摘要包含代码转换短语。我们参照现有方法（包括流水线式、端到端和零样本方法）进行了性能评估。研究表明，将现有资源作为预训练步骤并不能提升CroCoSum上的性能，表明现有资源的泛化能力有限。最后，我们通过定性错误分析讨论了在代码转换生成任务中评估跨语言摘要器所面临的挑战。我们的数据集和代码可通过https://github.com/RosenZhang/CroCoSum获取。