Adaptive exploration methods propose ways to learn complex policies via alternating between exploration and exploitation. An important question for such methods is to determine the appropriate moment to switch between exploration and exploitation and vice versa. This is critical in domains that require the learning of long and complex sequences of actions. In this work, we present a generic adaptive exploration framework that employs uncertainty to address this important issue in a principled manner. Our framework includes previous adaptive exploration approaches as special cases. Moreover, we can incorporate in our framework any uncertainty-measuring mechanism of choice, for instance mechanisms used in intrinsic motivation or epistemic uncertainty-based exploration methods. We experimentally demonstrate that our framework gives rise to adaptive exploration strategies that outperform standard ones across several environments.

翻译：自适应探索方法提出了通过交替进行探索与利用来学习复杂策略的途径。此类方法面临的一个关键问题在于确定何时在探索与利用之间进行切换，反之亦然。这在需要学习长而复杂动作序列的领域中尤为重要。本研究提出了一种通用的自适应探索框架，该框架以不确定性为驱动，以系统化的方式解决这一重要问题。我们的框架将先前的自适应探索方法涵盖为特例。此外，该框架可整合任意选择的不确定性度量机制，例如内在激励或基于认知不确定性的探索方法所采用的机制。实验结果表明，我们的框架所产生的自适应探索策略在多种环境中均优于标准方法。