Given a matrix $\mathbf{A} \in \mathbb{R}^{k \times n}$, a partitioning of $[k]$ into groups $S_1,\dots,S_m$, an outer norm $p$, and a collection of inner norms such that either $p \ge 1$ and $p_1,\dots,p_m \ge 2$ or $p_1=\dots=p_m=p \ge 1/\log n$, we prove that there is a sparse weight vector $\mathbf{\beta} \in \mathbb{R}^{m}$ such that $\sum_{i=1}^m \mathbf{\beta}_i \cdot \|\mathbf{A}_{S_i}\mathbf{x}\|_{p_i}^p \approx_{1\pm\varepsilon} \sum_{i=1}^m \|\mathbf{A}_{S_i}\mathbf{x}\|_{p_i}^p$, where the number of nonzero entries of $\mathbf{\beta}$ is at most $O_{p,p_i}(\varepsilon^{-2}n^{\max(1,p/2)}(\log n)^2(\log(n/\varepsilon)))$. When $p_1\dots,p_m \ge 2$, this weight vector arises from an importance sampling procedure based on the \textit{block Lewis weights}, a recently proposed generalization of Lewis weights. Additionally, we prove that there exist efficient algorithms to find the sparse weight vector $\mathbf{\beta}$ in several important regimes of $p$ and $p_1,\dots,p_m$. Our results imply a $\widetilde{O}(\varepsilon^{-1}\sqrt{n})$-linear system solve iteration complexity for the problem of minimizing sums of Euclidean norms, improving over the previously known $\widetilde{O}(\sqrt{m}\log({1/\varepsilon}))$ iteration complexity when $m \gg n$. Our main technical contribution is a substantial generalization of the \textit{change-of-measure} method that Bourgain, Lindenstrauss, and Milman used to obtain the analogous result when every group has size $1$. Our generalization allows one to analyze change of measures beyond those implied by D. Lewis's original construction, including the measure implied by the block Lewis weights and natural approximations of this measure.
翻译:给定矩阵 $\mathbf{A} \in \mathbb{R}^{k \times n}$、$[k]$ 划分为 $S_1,\dots,S_m$ 的分组、外部范数 $p$,以及一组内部范数满足 $p \ge 1$ 且 $p_1,\dots,p_m \ge 2$ 或 $p_1=\dots=p_m=p \ge 1/\log n$,我们证明存在稀疏权重向量 $\mathbf{\beta} \in \mathbb{R}^{m}$ 使得 $\sum_{i=1}^m \mathbf{\beta}_i \cdot \|\mathbf{A}_{S_i}\mathbf{x}\|_{p_i}^p \approx_{1\pm\varepsilon} \sum_{i=1}^m \|\mathbf{A}_{S_i}\mathbf{x}\|_{p_i}^p$,其中 $\mathbf{\beta}$ 的非零项数量至多为 $O_{p,p_i}(\varepsilon^{-2}n^{\max(1,p/2)}(\log n)^2(\log(n/\varepsilon)))$。当 $p_1\dots,p_m \ge 2$ 时,该权重向量源于基于\textit{块Lewis权重}的重要性采样过程,这是Lewis权重最近提出的推广形式。此外,我们证明在 $p$ 和 $p_1,\dots,p_m$ 的若干重要取值范围内,存在高效算法可求解该稀疏权重向量 $\mathbf{\beta}$。我们的结果意味着欧几里得范数求和最小化问题具有 $\widetilde{O}(\varepsilon^{-1}\sqrt{n})$ 的线性系统求解迭代复杂度,当 $m \gg n$ 时改进了先前已知的 $\widetilde{O}(\sqrt{m}\log({1/\varepsilon}))$ 迭代复杂度。我们的主要技术贡献是对Bourgain、Lindenstrauss和Milman用于证明每组规模为$1$时相应结果的\textit{测度变换方法}进行了实质性推广。该推广使得我们能够分析超越D. Lewis原始构造所隐含测度的测度变换,包括块Lewis权重隐含的测度及其自然近似形式。