High sample complexity has long been a challenge for RL. On the other hand, humans learn to perform tasks not only from interaction or demonstrations, but also by reading unstructured text documents, e.g., instruction manuals. Instruction manuals and wiki pages are among the most abundant data that could inform agents of valuable features and policies or task-specific environmental dynamics and reward structures. Therefore, we hypothesize that the ability to utilize human-written instruction manuals to assist learning policies for specific tasks should lead to a more efficient and better-performing agent. We propose the Read and Reward framework. Read and Reward speeds up RL algorithms on Atari games by reading manuals released by the Atari game developers. Our framework consists of a QA Extraction module that extracts and summarizes relevant information from the manual and a Reasoning module that evaluates object-agent interactions based on information from the manual. Auxiliary reward is then provided to a standard A2C RL agent, when interaction is detected. When assisted by our design, A2C improves on 4 games in the Atari environment with sparse rewards, and requires 1000x less training frames compared to the previous SOTA Agent 57 on Skiing, the hardest game in Atari.

翻译：长期以来，高样本复杂度一直是强化学习的挑战。另一方面，人类不仅通过交互或演示学习执行任务，还通过阅读非结构化文本文档（例如说明手册）来学习。说明手册和维基百科页面是最丰富的数据来源，它们可以告知智能体有价值的特征和策略，或任务特定的环境动态和奖励结构。因此，我们假设利用人类编写的说明手册来辅助学习特定任务的策略，应能使智能体更高效且性能更优。我们提出了“阅读与奖励”框架。该框架通过阅读雅达利游戏开发者发布的说明手册，加速了在雅达利游戏上的强化学习算法。我们的框架包含一个问答提取模块，用于从手册中提取和总结相关信息；以及一个推理模块，用于基于手册信息评估对象-智能体交互。当检测到交互时，会向标准的A2C强化学习智能体提供辅助奖励。在我们的设计辅助下，A2C在雅达利环境中稀疏奖励的4个游戏上表现提升，并且在雅达利最困难的游戏Skiing上，所需的训练帧数比之前的最优模型Agent 57少1000倍。