Overparameterized models have recently challenged conventional learning theory by exhibiting improved generalization beyond the interpolation limit, a phenomenon known as benign overfitting. This work introduces Adaptive Benign Overfitting (ABO), extending the recursive least-squares (RLS) framework to this regime through a numerically stable formulation based on orthogonal-triangular updates. A QR-based exponentially weighted RLS (QR-EWRLS) algorithm is introduced, combining random Fourier feature mappings with forgetting-factor regularization to enable online adaptation under non-stationary conditions. The orthogonal decomposition prevents the numerical divergence associated with covariance-form RLS while retaining adaptability to evolving data distributions. Experiments on nonlinear synthetic time series confirm that the proposed approach maintains bounded residuals and stable condition numbers while reproducing the double-descent behavior characteristic of overparameterized models. Applications to forecasting foreign exchange and electricity demand show that ABO is highly accurate (comparable to baseline kernel methods) while achieving speed improvements of between 20 and 40 percent. The results provide a unified view linking adaptive filtering, kernel approximation, and benign overfitting within a stable online learning framework.

翻译：过参数化模型近期挑战了传统学习理论，其表现出的超越插值极限的泛化提升现象被称为良性过拟合。本文提出自适应良性过拟合（ABO），通过基于正交三角更新的数值稳定公式，将递归最小二乘（RLS）框架扩展至该机制。我们引入一种基于QR分解的指数加权RLS（QR-EWRLS）算法，该算法将随机傅里叶特征映射与遗忘因子正则化相结合，以实现非平稳条件下的在线自适应。正交分解避免了协方差形式RLS相关的数值发散问题，同时保持了对演化数据分布的适应能力。在非线性合成时间序列上的实验证实，所提方法在复现过参数化模型特有的双下降行为的同时，保持了有界残差和稳定的条件数。在外汇预测和电力需求预测中的应用表明，ABO在保持高精度（与基线核方法相当）的同时，实现了20%至40%的速度提升。这些结果为在稳定的在线学习框架内统一连接自适应滤波、核近似与良性过拟合提供了整体视角。