Updating a truncated Singular Value Decomposition (SVD) is crucial in representation learning, especially when dealing with large-scale data matrices that continuously evolve in practical scenarios. Aligning SVD-based models with fast-paced updates becomes increasingly important. Existing methods for updating truncated SVDs employ Rayleigh-Ritz projection procedures, where projection matrices are augmented based on original singular vectors. However, these methods suffer from inefficiency due to the densification of the update matrix and the application of the projection to all singular vectors. To address these limitations, we introduce a novel method for dynamically approximating the truncated SVD of a sparse and temporally evolving matrix. Our approach leverages sparsity in the orthogonalization process of augmented matrices and utilizes an extended decomposition to independently store projections in the column space of singular vectors. Numerical experiments demonstrate a remarkable efficiency improvement of an order of magnitude compared to previous methods. Remarkably, this improvement is achieved while maintaining a comparable precision to existing approaches.

翻译：在表示学习中，更新截断奇异值分解（SVD）至关重要，尤其是在实际场景中处理持续演化的大规模数据矩阵时。使基于SVD的模型适应快速更新变得日益重要。现有更新截断SVD的方法采用Rayleigh-Ritz投影过程，其中投影矩阵基于原始奇异向量进行增广。然而，这些方法因更新矩阵的稠密化以及对所有奇异向量应用投影而导致效率低下。为克服这些局限，我们提出了一种新方法，用于动态逼近稀疏且随时间演化的矩阵的截断SVD。该方法在增广矩阵的正交化过程中利用稀疏性，并采用扩展分解将投影独立存储在奇异向量列空间中。数值实验表明，与先前方法相比，我们的方法实现了显著的效率提升（达到一个数量级）。值得注意的是，在保持与现有方法相当精度的同时，这一改进得以实现。