We present the framework of slowly varying regression under sparsity, allowing sparse regression models to exhibit slow and sparse variations. The problem of parameter estimation is formulated as a mixed-integer optimization problem. We demonstrate that it can be precisely reformulated as a binary convex optimization problem through a novel relaxation technique. This relaxation involves a new equality on Moore-Penrose inverses, convexifying the non-convex objective function while matching the original objective on all feasible binary points. This enables us to efficiently solve the problem to provable optimality using a cutting plane-type algorithm. We develop a highly optimized implementation of this algorithm, substantially improving upon the asymptotic computational complexity of a straightforward implementation. Additionally, we propose a fast heuristic method that guarantees a feasible solution and, as empirically illustrated, produces high-quality warm-start solutions for the binary optimization problem. To tune the framework's hyperparameters, we suggest a practical procedure relying on binary search that, under certain assumptions, is guaranteed to recover the true model parameters. On both synthetic and real-world datasets, we demonstrate that the resulting algorithm outperforms competing formulations in comparable times across various metrics, including estimation accuracy, predictive power, and computational time. The algorithm is highly scalable, allowing us to train models with thousands of parameters. Our implementation is available open-source at https://github.com/vvdigalakis/SSVRegression.git.

翻译：我们提出稀疏性下的缓慢变化回归框架，允许稀疏回归模型呈现缓慢且稀疏的变化。参数估计问题被构建为混合整数优化问题。我们证明通过一种新颖的松弛技术，该问题可以精确重构为二元凸优化问题。该松弛涉及关于Moore-Penrose逆的新等式，在凸化非凸目标函数的同时，使所有可行二元点上的原目标函数保持不变。这使我们能够利用割平面类算法高效地求解问题至可证明的最优性。我们开发了该算法的高度优化实现，显著改进了朴素实现的渐近计算复杂度。此外，我们提出一种快速启发式方法，该方法保证得到可行解，且如实验所示，能为二元优化问题生成高质量的热启动初始解。为调整框架的超参数，我们提出一种基于二分搜索的实用程序，该程序在特定假设下保证能恢复真实模型参数。在合成数据集和真实世界数据集上，我们证明所得算法在可比时间内，于估计精度、预测能力和计算时间等多个指标上均优于竞争性方法。该算法具有高度可扩展性，能够训练包含数千个参数的模型。我们的开源实现见https://github.com/vvdigalakis/SSVRegression.git。