Linearly parametrized models are widely used in control and signal processing, with the least-squares (LS) estimate being the archetypical solution. When the input is insufficiently exciting, the LS problem may be unsolvable or numerically unstable. This issue can be resolved through regularization, typically with ridge regression. Although regularized estimators reduce the variance error, it remains important to quantify their estimation uncertainty. A possible approach for linear regression is to construct confidence ellipsoids with the Sign-Perturbed Sums (SPS) ellipsoidal outer approximation (EOA) algorithm. The SPS EOA builds non-asymptotic confidence ellipsoids under the assumption that the noises are independent and symmetric about zero. This paper introduces an extension of the SPS EOA algorithm to ridge regression, and derives probably approximately correct (PAC) upper bounds for the resulting region sizes. Compared with previous analyses, our result explicitly show how the regularization parameter affects the region sizes, and provide tighter bounds under weaker excitation assumptions. Finally, the practical effect of regularization is also demonstrated via simulation experiments.

翻译：线性参数化模型在控制和信号处理领域应用广泛，其中最小二乘（LS）估计是典型解法。当输入激励不足时，LS问题可能无解或数值不稳定。该问题可通过正则化方法解决，通常采用岭回归。虽然正则化估计量能减小方差误差，量化其估计不确定性仍然至关重要。对于线性回归，一种可行方案是使用符号扰动和（SPS）椭球外逼近（EOA）算法构建置信椭球。SPS EOA算法在噪声独立且关于零对称的假设下构建非渐近置信椭球。本文提出将SPS EOA算法扩展至岭回归，并为所得区域尺寸推导出概率近似正确（PAC）上界。与先前分析相比，我们的结果明确展示了正则化参数如何影响区域尺寸，并在更弱的激励假设下给出更紧的界。最后，通过仿真实验验证了正则化的实际效果。