Large Language Models (LLMs) have demonstrated impressive capabilities in creative tasks such as storytelling and E-mail generation. However, as LLMs are primarily trained on final text results rather than intermediate revisions, it might be challenging for them to perform text rewriting tasks. Most studies in the rewriting tasks focus on a particular transformation type within the boundaries of single sentences. In this work, we develop new strategies for instruction tuning and reinforcement learning to better align LLMs for cross-sentence rewriting tasks using diverse wording and structures expressed through natural languages including 1) generating rewriting instruction data from Wiki edits and public corpus through instruction generation and chain-of-thought prompting; 2) collecting comparison data for reward model training through a new ranking function. To facilitate this research, we introduce OpenRewriteEval, a novel benchmark covers a wide variety of rewriting types expressed through natural language instructions. Our results show significant improvements over a variety of baselines. The public repository is available on GitHub under Google Research (https://github.com/google-research/google-research/tree/master/rewritelm).

翻译：大语言模型在故事创作和邮件生成等创意任务中展现出令人印象深刻的能力。然而，由于大语言模型主要基于最终文本结果而非中间修订过程进行训练，它们在执行文本重写任务时可能面临挑战。现有重写任务研究大多聚焦于单句范围内的特定转换类型。本研究开发了新的指令微调和强化学习策略，通过自然语言表达的不同词汇与结构，更好地对齐大语言模型进行跨句子重写任务，具体包括：1）通过指令生成和思维链提示，从维基编辑和公开语料库生成重写指令数据；2）通过新型排序函数收集用于奖励模型训练的对比数据。为促进该研究，我们提出了OpenRewriteEval这一新型基准测试，涵盖通过自然语言指令表达的各种重写类型。实验结果表明，本方法在多个基线上取得了显著改进。相关公共代码仓库已发布在谷歌研究GitHub页面（https://github.com/google-research/google-research/tree/master/rewritelm）。