Users interact with text, image, code, or other editors on a daily basis. However, machine learning models are rarely trained in the settings that reflect the interactivity between users and their editor. This is understandable as training AI models with real users is not only slow and costly, but what these models learn may be specific to user interface design choices. Unfortunately, this means most of the research on text, code, and image generation has focused on non-interactive settings, whereby the model is expected to get everything right without accounting for any input from a user who may be willing to help. We introduce a new Interactive Text Generation task that allows training generation models interactively without the costs of involving real users, by using user simulators that provide edits that guide the model towards a given target text. We train our interactive models using Imitation Learning, and our experiments against competitive non-interactive generation models show that models trained interactively are superior to their non-interactive counterparts, even when all models are given the same budget of user inputs or edits.

翻译：用户日常与文本、图像、代码或其他编辑器进行交互。然而，机器学习模型很少在反映用户与编辑器之间交互性的设置中进行训练。这可以理解，因为用真实用户训练AI模型不仅缓慢且成本高昂，而且这些模型学到的东西可能局限于用户界面设计的选择。不幸的是，这意味着大多数关于文本、代码和图像生成的研究都集中在非交互式设置上，即模型期望在无需考虑可能愿意提供帮助的用户输入的情况下，将所有内容一次性生成正确。我们提出了一项新的交互式文本生成任务，该任务通过使用提供编辑引导的用户模拟器（模拟器会提供编辑内容以引导模型朝向给定的目标文本），在不涉及真实用户成本的情况下交互式地训练生成模型。我们使用模仿学习来训练交互式模型，在与具有竞争力的非交互式生成模型进行的实验中表明，即使所有模型获得相同预算的用户输入或编辑次数，交互式训练的模型也优于其非交互式对应物。