Adaptive exploration methods propose ways to learn complex policies via alternating between exploration and exploitation. An important question for such methods is to determine the appropriate moment to switch between exploration and exploitation and vice versa. This is critical in domains that require the learning of long and complex sequences of actions. In this work, we present a generic adaptive exploration framework that employs uncertainty to address this important issue in a principled manner. Our framework includes previous adaptive exploration approaches as special cases. Moreover, we can incorporate in our framework any uncertainty-measuring mechanism of choice, for instance mechanisms used in intrinsic motivation or epistemic uncertainty-based exploration methods. We experimentally demonstrate that our framework gives rise to adaptive exploration strategies that outperform standard ones across several environments.

翻译：自适应探索方法提出了通过交替进行探索与利用来学习复杂策略的途径。这类方法的一个关键问题在于如何确定在探索与利用之间切换的适宜时机，这对于需要学习长序列复杂动作的领域尤为重要。本研究提出了一种通用的自适应探索框架，该框架以不确定性为驱动，以系统化的方式解决这一重要问题。我们的框架将以往的自适应探索方法纳入作为特例。此外，框架可兼容任选的不确定性度量机制，例如内在激励或基于认知不确定性的探索方法所采用的机制。实验结果表明，本框架衍生的自适应探索策略在多种环境中均优于传统方法。