How can we design Natural Language Processing (NLP) systems that learn from human feedback? There is a growing research body of Human-in-the-loop (HITL) NLP frameworks that continuously integrate human feedback to improve the model itself. HITL NLP research is nascent but multifarious -- solving various NLP problems, collecting diverse feedback from different people, and applying different methods to learn from collected feedback. We present a survey of HITL NLP work from both Machine Learning (ML) and Human-Computer Interaction (HCI) communities that highlights its short yet inspiring history, and thoroughly summarize recent frameworks focusing on their tasks, goals, human interactions, and feedback learning methods. Finally, we discuss future directions for integrating human feedback in the NLP development loop.

翻译：我们如何设计从人类反馈中学习的自然语言处理系统? 人类流动中的人(HITL)国家语言处理框架的研究机构不断增多,不断整合人类反馈,以改善模型本身。 HITL NLP研究初创而多样 -- -- 解决各种非语言处理问题,收集不同人群的不同反馈,并采用不同方法从所收集的反馈中学习。我们从机器学习(ML)和人-计算机互动(HCI)两个社区展示了HITL NLP工作的调查,其中突出介绍了其短暂但鼓舞人心的历史,并透彻总结了侧重于其任务、目标、人类互动和反馈学习方法的近期框架。最后,我们讨论了将人类反馈纳入国家语言学习方案发展循环的未来方向。