Adaptive exploration methods propose ways to learn complex policies via alternating between exploration and exploitation. An important question for such methods is to determine the appropriate moment to switch between exploration and exploitation and vice versa. This is critical in domains that require the learning of long and complex sequences of actions. In this work, we present a generic adaptive exploration framework that employs uncertainty to address this important issue in a principled manner. Our framework includes previous adaptive exploration approaches as special cases. Moreover, we can incorporate in our framework any uncertainty-measuring mechanism of choice, for instance mechanisms used in intrinsic motivation or epistemic uncertainty-based exploration methods. We experimentally demonstrate that our framework gives rise to adaptive exploration strategies that outperform standard ones across several environments.

翻译：自适应探索方法提出了通过交替进行探索与利用来学习复杂策略的途径。此类方法面临的一个关键问题在于如何确定在探索与利用之间切换的适宜时机，这在需要学习长序列复杂动作的领域中尤为重要。本研究提出了一种通用的自适应探索框架，该框架以不确定性为驱动，以系统化方式解决这一核心问题。我们的框架将既往的自适应探索方法涵盖为特例，并能够整合任选的不确定性度量机制，例如内在激励或认知不确定性探索方法中采用的机制。实验结果表明，本框架衍生的自适应探索策略在多种测试环境中均优于传统方法。