We present a representation-driven framework for reinforcement learning. By representing policies as estimates of their expected values, we leverage techniques from contextual bandits to guide exploration and exploitation. Particularly, embedding a policy network into a linear feature space allows us to reframe the exploration-exploitation problem as a representation-exploitation problem, where good policy representations enable optimal exploration. We demonstrate the effectiveness of this framework through its application to evolutionary and policy gradient-based approaches, leading to significantly improved performance compared to traditional methods. Our framework provides a new perspective on reinforcement learning, highlighting the importance of policy representation in determining optimal exploration-exploitation strategies.

翻译：我们提出了一种表示驱动的强化学习框架。通过将策略表示为其期望值的估计，我们利用上下文赌博机技术来指导探索与利用。特别是，将策略网络嵌入线性特征空间使我们能够将探索-利用问题重新构建为表示-利用问题，其中良好的策略表示能够实现最优探索。我们通过将该框架应用于进化方法和基于策略梯度的技术，展示了其有效性，相比传统方法实现了显著的性能提升。该框架为强化学习提供了新视角，凸显了策略表示在确定最优探索-利用策略中的关键作用。