Thompson sampling (TS) is a Bayesian randomized exploration strategy that samples options (e.g., system parameters or control laws) from the current posterior and then applies the selected option that is optimal for a task, thereby balancing exploration and exploitation; this makes TS effective for active learning-based controller design. However, TS relies on finite parametric representations, which limits its applicability to more general spaces, which are more commonly encountered in control system design. To address this issue, this work proposes a parameterization method for control law learning using reproducing kernel Hilbert spaces and designs a data-driven active learning control approach. Specifically, the proposed method treats the control law as an element in a function space, allowing the design of control laws without imposing restrictions on the system structure or the form of the controller. A TS framework is proposed in this work to reduce control costs through online exploration and exploitation, and the convergence guarantees are further provided for the learning process. Theoretical analysis shows that the proposed method learns the relationship between control laws and closed-loop performance metrics at an exponential rate, and the upper bound of control regret is also derived. Furthermore, the closed-loop stability of the proposed learning framework is analyzed. Numerical experiments on controlling unknown nonlinear systems validate the effectiveness of the proposed method.

翻译：汤普森采样（TS）是一种贝叶斯随机探索策略，它从当前后验分布中采样选项（例如系统参数或控制律），然后应用对任务最优的选定选项，从而平衡探索与利用；这使得TS对于基于主动学习的控制器设计十分有效。然而，TS依赖于有限的参数化表示，这限制了其在更一般空间（在控制系统设计中更为常见）中的适用性。为解决此问题，本文提出了一种利用再生核希尔伯特空间的控制律学习参数化方法，并设计了一种数据驱动的主动学习控制方法。具体而言，所提方法将控制律视为函数空间中的一个元素，从而允许在不强加系统结构或控制器形式限制的情况下设计控制律。本文提出了一个TS框架，通过在线探索与利用来降低控制成本，并进一步为学习过程提供了收敛性保证。理论分析表明，所提方法以指数速率学习控制律与闭环性能指标之间的关系，并推导了控制遗憾的上界。此外，本文分析了所提学习框架的闭环稳定性。在控制未知非线性系统上的数值实验验证了所提方法的有效性。