Users interact with text, image, code, or other editors on a daily basis. However, machine learning models are rarely trained in the settings that reflect the interactivity between users and their editor. This is understandable as training AI models with real users is not only slow and costly, but what these models learn may be specific to user interface design choices. Unfortunately, this means most of the research on text, code, and image generation has focused on non-interactive settings, whereby the model is expected to get everything right without accounting for any input from a user who may be willing to help. We introduce a new Interactive Text Generation task that allows training generation models interactively without the costs of involving real users, by using user simulators that provide edits that guide the model towards a given target text. We train our interactive models using Imitation Learning, and our experiments against competitive non-interactive generation models show that models trained interactively are superior to their non-interactive counterparts, even when all models are given the same budget of user inputs or edits.

翻译：用户日常与文本、图像、代码或其他编辑器进行交互，但机器学习模型很少在反映用户与编辑器之间交互性的环境中进行训练。这可以理解，因为用真实用户训练人工智能模型不仅缓慢且成本高昂，而且模型学习到的内容可能特定于用户界面设计选择。遗憾的是，这意味着大多数关于文本、代码和图像生成的研究都聚焦于非交互式场景——即模型被期望在不考虑可能愿意提供帮助的用户的任何输入的情况下，一次性生成正确结果。我们提出了一项新的交互式文本生成任务，通过使用用户模拟器提供引导模型朝向给定目标文本的编辑操作，无需涉及真实用户的成本即可交互式地训练生成模型。我们采用模仿学习训练交互式模型，与具有竞争力的非交互式生成模型进行的实验表明，即使所有模型获得相同数量的用户输入或编辑预算，交互式训练的模型仍显著优于非交互式对应模型。