In this paper, we study the following nonlinear matrix decomposition (NMD) problem: given a sparse nonnegative matrix $X$, find a low-rank matrix $\Theta$ such that $X \approx f(\Theta)$, where $f$ is an element-wise nonlinear function. We focus on the case where $f(\cdot) = \max(0, \cdot)$, the rectified unit (ReLU) non-linear activation. We refer to the corresponding problem as ReLU-NMD. We first provide a brief overview of the existing approaches that were developed to tackle ReLU-NMD. Then we introduce two new algorithms: (1) aggressive accelerated NMD (A-NMD) which uses an adaptive Nesterov extrapolation to accelerate an existing algorithm, and (2) three-block NMD (3B-NMD) which parametrizes $\Theta = WH$ and leads to a significant reduction in the computational cost. We also propose an effective initialization strategy based on the nuclear norm as a proxy for the rank function. We illustrate the effectiveness of the proposed algorithms (available on gitlab) on synthetic and real-world data sets.

翻译：本文研究如下非线性矩阵分解（NMD）问题：给定稀疏非负矩阵 $X$，寻找低秩矩阵 $\Theta$，使得 $X \approx f(\Theta)$，其中 $f$ 为逐元素非线性函数。我们聚焦于 $f(\cdot) = \max(0, \cdot)$ 即整流线性单元（ReLU）非线性激活的情形，并将相应问题称为ReLU-NMD。首先简要概述现有解决ReLU-NMD的方法，随后提出两种新算法：(1) 积极加速NMD（A-NMD），采用自适应Nesterov外推法加速现有算法；(2) 三块NMD（3B-NMD），通过参数化 $\Theta = WH$ 显著降低计算成本。我们还提出基于核范数（作为秩函数的代理）的有效初始化策略。在合成数据集与真实数据集上的实验（代码见gitlab）验证了所提算法的有效性。