TWe establish regret lower bounds for adaptively controlling an unknown linear Gaussian system with quadratic costs. We combine ideas from experiment design, estimation theory and a perturbation bound of certain information matrices to derive regret lower bounds exhibiting scaling on the order of magnitude $\sqrt{T}$ in the time horizon $T$. Our bounds accurately capture the role of control-theoretic parameters and we are able to show that systems that are hard to control are also hard to learn to control; when instantiated to state feedback systems we recover the dimensional dependency of earlier work but with improved scaling with system-theoretic constants such as system costs and Gramians. Furthermore, we extend our results to a class of partially observed systems and demonstrate that systems with poor observability structure also are hard to learn to control.
翻译:本文针对具有二次成本的未知线性高斯系统的自适应控制问题,建立了遗憾下界。我们结合实验设计、估计理论以及特定信息矩阵的扰动界等思想,推导出在时间范围 $T$ 内具有 $\sqrt{T}$ 量级尺度的遗憾下界。我们的下界精确地捕捉了控制理论参数的作用,并且能够证明难以控制的系统同样难以学习控制;当应用于状态反馈系统时,我们重现了早期工作的维度依赖性,但在系统理论常数(如系统成本和格拉姆矩阵)的尺度关系上有所改进。此外,我们将结果推广到一类部分可观测系统,并证明具有较差可观测性结构的系统同样难以学习控制。