Adaptive exploration methods propose ways to learn complex policies via alternating between exploration and exploitation. An important question for such methods is to determine the appropriate moment to switch between exploration and exploitation and vice versa. This is critical in domains that require the learning of long and complex sequences of actions. In this work, we present a generic adaptive exploration framework that employs uncertainty to address this important issue in a principled manner. Our framework includes previous adaptive exploration approaches as special cases. Moreover, we can incorporate in our framework any uncertainty-measuring mechanism of choice, for instance mechanisms used in intrinsic motivation or epistemic uncertainty-based exploration methods. We experimentally demonstrate that our framework gives rise to adaptive exploration strategies that outperform standard ones across several environments.

翻译：自适应探索方法提出了通过交替进行探索与利用来学习复杂策略的途径。此类方法面临的一个重要问题是确定在探索与利用之间切换（反之亦然）的恰当时机。这在需要学习长且复杂动作序列的领域中尤为关键。本研究提出了一种通用的自适应探索框架，该框架以不确定性为驱动，以原则性方式解决这一重要问题。我们的框架将先前的自适应探索方法纳入为特例。此外，我们能够在框架中整合任意选择的不确定性度量机制，例如内在动机或基于认知不确定性的探索方法所采用的机制。实验结果表明，我们的框架所产生的自适应探索策略在多种环境中均优于标准方法。