Automatic Speech Recognition (ASR) transcripts exhibit recognition errors and various spoken language phenomena such as disfluencies, ungrammatical sentences, and incomplete sentences, hence suffering from poor readability. To improve readability, we propose a Contextualized Spoken-to-Written conversion (CoS2W) task to address ASR and grammar errors and also transfer the informal text into the formal style with content preserved, utilizing contexts and auxiliary information. This task naturally matches the in-context learning capabilities of Large Language Models (LLMs). To facilitate comprehensive comparisons of various LLMs, we construct a document-level Spoken-to-Written conversion of ASR Transcripts Benchmark (SWAB) dataset. Using SWAB, we study the impact of different granularity levels on the CoS2W performance, and propose methods to exploit contexts and auxiliary information to enhance the outputs. Experimental results reveal that LLMs have the potential to excel in the CoS2W task, particularly in grammaticality and formality, our methods achieve effective understanding of contexts and auxiliary information by LLMs. We further investigate the effectiveness of using LLMs as evaluators and find that LLM evaluators show strong correlations with human evaluations on rankings of faithfulness and formality, which validates the reliability of LLM evaluators for the CoS2W task.

翻译：自动语音识别（ASR）转录文本存在识别错误及多种口语现象，如不流畅表达、非语法句和不完整句，因而可读性较差。为提升可读性，我们提出一种基于上下文的口语到书面语转换任务，旨在利用上下文及辅助信息纠正ASR与语法错误，同时将非正式文本转换为内容保留的正式文体。该任务天然契合大语言模型的上下文学习能力。为系统比较各类大语言模型，我们构建了文档级ASR转录口语到书面语转换基准数据集。基于该数据集，我们研究了不同粒度层级对转换性能的影响，并提出利用上下文与辅助信息提升输出质量的方法。实验结果表明，大语言模型在转换任务中展现出显著潜力，尤其在语法规范性与文体正式性方面；我们提出的方法能有效帮助大语言模型理解上下文与辅助信息。我们进一步探究了大语言模型作为评估器的有效性，发现其在忠实度与正式性排序任务中与人工评估结果具有强相关性，验证了大语言模型评估器在转换任务中的可靠性。