Candecomp / PARAFAC (CP) decomposition, a generalization of the matrix singular value decomposition to higher-dimensional tensors, is a popular tool for analyzing multidimensional sparse data. On tensors with billions of nonzero entries, computing a CP decomposition is a computationally intensive task. We propose the first distributed-memory implementations of two randomized CP decomposition algorithms, CP-ARLS-LEV and STS-CP, that offer nearly an order-of-magnitude speedup at high decomposition ranks over well-tuned non-randomized decomposition packages. Both algorithms rely on leverage score sampling and enjoy strong theoretical guarantees, each with varying time and accuracy tradeoffs. We tailor the communication schedule for our random sampling algorithms, eliminating expensive reduction collectives and forcing communication costs to scale with the random sample count. Finally, we optimize the local storage format for our methods, switching between analogues of compressed sparse column and compressed sparse row formats. Experiments show that our methods are fast and scalable, producing 11x speedup over SPLATT by decomposing the billion-scale Reddit tensor on 512 CPU cores in under two minutes.

翻译：Candecomp/PARAFAC（CP）分解是矩阵奇异值分解在高维张量上的推广，是分析多维稀疏数据的常用工具。对于包含数十亿非零元素的张量，计算CP分解是一项计算密集型的任务。我们提出了两种随机CP分解算法——CP-ARLS-LEV和STS-CP的首个分布式内存实现，这些算法在高分解秩下相比经过良好调优的非随机分解包可实现近一个数量级的加速。两种算法均依赖杠杆分数采样，并享有强理论保证，在时间和精度权衡上各有差异。我们针对随机采样算法定制了通信调度，消除了昂贵的归约集合操作，使通信成本随随机样本数量扩展。最后，我们优化了方法的本地存储格式，在压缩稀疏列格式和压缩稀疏行格式的类似形式之间切换。实验表明，我们的方法快速且可扩展，在512个CPU核心上分解十亿规模的Reddit张量仅需不到两分钟，相比SPLATT实现了11倍加速。