Across scientific disciplines, Laplacian eigenvectors serve as a fundamental basis for simplifying complex systems, from signal processing to quantum mechanics. In reinforcement learning (RL), these eigenvectors provide a natural basis for approximating reward functions; however, their use is typically limited to their linear span, which restricts expressivity in complex environments. We introduce the Laplacian Keyboard (LK), a hierarchical framework that goes beyond the linear span. LK constructs a task-agnostic library of options from these eigenvectors, forming a behavior basis guaranteed to contain the optimal policy for any reward within the linear span. A meta-policy learns to stitch these options dynamically, enabling efficient learning of policies outside the original linear constraints. We establish theoretical bounds on zero-shot approximation error and demonstrate empirically that LK surpasses zero-shot solutions while achieving improved sample efficiency compared to standard RL methods.

翻译：在科学各领域中，拉普拉斯特征向量作为简化复杂系统的基础工具，从信号处理到量子力学均有广泛应用。在强化学习（RL）中，这些特征向量为逼近奖励函数提供了自然基；然而，其使用通常局限于线性张成空间，这在复杂环境中限制了表达能力。我们提出拉普拉斯键盘（LK），一种超越线性张成空间的层次化框架。LK从这些特征向量构建任务无关的选项库，形成保证包含线性张成空间内任意奖励对应最优策略的行为基。元策略通过动态组合这些选项进行学习，从而能够高效学习超出原始线性约束的策略。我们建立了零样本逼近误差的理论界，并通过实验证明LK在超越零样本解法的同时，相比标准RL方法实现了更高的样本效率。