Deep Generative AI has been a long-standing essential topic in the machine learning community, which can impact a number of application areas like text generation and computer vision. The major paradigm to train a generative model is maximum likelihood estimation, which pushes the learner to capture and approximate the target data distribution by decreasing the divergence between the model distribution and the target distribution. This formulation successfully establishes the objective of generative tasks, while it is incapable of satisfying all the requirements that a user might expect from a generative model. Reinforcement learning, serving as a competitive option to inject new training signals by creating new objectives that exploit novel signals, has demonstrated its power and flexibility to incorporate human inductive bias from multiple angles, such as adversarial learning, hand-designed rules and learned reward model to build a performant model. Thereby, reinforcement learning has become a trending research field and has stretched the limits of generative AI in both model design and application. It is reasonable to summarize and conclude advances in recent years with a comprehensive review. Although there are surveys in different application areas recently, this survey aims to shed light on a high-level review that spans a range of application areas. We provide a rigorous taxonomy in this area and make sufficient coverage on various models and applications. Notably, we also surveyed the fast-developing large language model area. We conclude this survey by showing the potential directions that might tackle the limit of current models and expand the frontiers for generative AI.

翻译：深度生成式人工智能一直是机器学习领域至关重要的主题，其可影响文本生成和计算机视觉等多个应用领域。训练生成模型的主要范式是最大似然估计，它通过减小模型分布与目标分布之间的散度，推动学习器捕获并逼近目标数据分布。该公式成功构建了生成任务的目标，但无法满足用户对生成模型的所有期望。强化学习作为通过创建利用新型信号的新目标来注入新训练信号的竞争性选择，展示了其从多个角度（如对抗性学习、手工设计规则和习得奖励模型）融入人类归纳偏好的能力与灵活性，从而构建高性能模型。因此，强化学习已成为一个热门研究领域，并在模型设计和应用两方面拓展了生成式人工智能的边界。合理且必要地通过综合综述来总结近年的进展。尽管不同应用领域已有相关综述，但本综述旨在提供横跨多个应用领域的高层次回顾。我们建立了该领域的严格分类体系，并对各类模型和应用进行了充分覆盖。值得注意的是，我们亦综述了快速发展的大语言模型领域。最后，本综述指出了可能突破当前模型局限、拓展生成式人工智能前沿的潜在研究方向。