The $\ell_{1,\infty}$ norm is an efficient structured projection but the complexity of the best algorithm is unfortunately $\mathcal{O}\big(n m \log(n m)\big)$ for a matrix in $\mathbb{R}^{n\times m}$. In this paper, we propose a new bi-level projection method for which we show that the time complexity for the $\ell_{1,\infty}$ norm is only $\mathcal{O}\big(n m \big)$ for a matrix in $\mathbb{R}^{n\times m}$, and $\mathcal{O}\big(n + m \big)$ with full parallel power. We generalize our method to tensors and we propose a new multi-level projection, having an induced decomposition that yields a linear parallel speedup up to an exponential speedup factor, resulting in a time complexity lower-bounded by the sum of the dimensions. Experiments show that our bi-level $\ell_{1,\infty}$ projection is $2.5$ times faster than the actual fastest algorithm provided by \textit{Chu et. al.} while providing same accuracy and better sparsity in neural networks applications.

翻译：$\ell_{1,\infty}$范数是一种高效的结构化投影方法，但当前最优算法的时间复杂度对于$\mathbb{R}^{n\times m}$矩阵仍为$\mathcal{O}\big(n m \log(n m)\big)$。本文提出一种新型双层投影方法，证明其$\ell_{1,\infty}$范数的时间复杂度对于$\mathbb{R}^{n\times m}$矩阵仅为$\mathcal{O}\big(n m \big)$，在完全并行条件下可降至$\mathcal{O}\big(n + m \big)$。我们将该方法推广至张量，并提出新型多级投影方法，其诱导分解可实现线性至指数级加速因子，使得时间复杂度下界为各维度之和。实验表明，在神经网络应用中，我们的双层$\ell_{1,\infty}$投影相比\textit{Chu et al.}提出的当前最快算法速度提升2.5倍，且精度相同、稀疏性更优。