Weighted low rank approximation is a fundamental problem in numerical linear algebra, and it has many applications in machine learning. Given a matrix $M \in \mathbb{R}^{n \times n}$, a weight matrix $W \in \mathbb{R}_{\geq 0}^{n \times n}$, a parameter $k$, the goal is to output two matrices $U, V \in \mathbb{R}^{n \times k}$ such that $\| W \circ (M - U V^\top) \|_F$ is minimized, where $\circ$ denotes the Hadamard product. Such a problem is known to be NP-hard and even hard to approximate assuming Exponential Time Hypothesis [GG11, RSW16]. Meanwhile, alternating minimization is a good heuristic solution for approximating weighted low rank approximation. The work [LLR16] shows that, under mild assumptions, alternating minimization does provide provable guarantees. In this work, we develop an efficient and robust framework for alternating minimization. For weighted low rank approximation, this improves the runtime of [LLR16] from $n^2 k^2$ to $n^2k$. At the heart of our work framework is a high-accuracy multiple response regression solver together with a robust analysis of alternating minimization.

翻译：加权低秩逼近是数值线性代数中的基本问题，在机器学习领域具有广泛应用。给定矩阵$M \in \mathbb{R}^{n \times n}$、权重矩阵$W \in \mathbb{R}_{\geq 0}^{n \times n}$及参数$k$，目标是最小化$\| W \circ (M - U V^\top) \|_F$，其中$U, V \in \mathbb{R}^{n \times k}$为待求解矩阵，$\circ$表示哈达玛积。该问题已知为NP难问题，且根据指数时间假说[GG11, RSW16]甚至难以近似求解。交替最小化是解决加权低秩逼近问题的有效启发式方法。[LLR16]的研究表明，在温和假设条件下，交替最小化方法具有可证明的收敛保证。本文提出了一种高效且鲁棒的交替最小化框架。对于加权低秩逼近问题，该方法将[LLR16]的时间复杂度从$n^2 k^2$降至$n^2 k$。本框架的核心创新在于高精度多响应回归求解器与交替最小化鲁棒性分析的结合。