Collaborative filtering (CF) has been proven to be one of the most effective techniques for recommendation. Among all CF approaches, SimpleX is the state-of-the-art method that adopts a novel loss function and a proper number of negative samples. However, there is no work that optimizes SimpleX on multi-core CPUs, leading to limited performance. To this end, we perform an in-depth profiling and analysis of existing SimpleX implementations and identify their performance bottlenecks including (1) irregular memory accesses, (2) unnecessary memory copies, and (3) redundant computations. To address these issues, we propose an efficient CF training system (called HEAT) that fully enables the multi-level caching and multi-threading capabilities of modern CPUs. Specifically, the optimization of HEAT is threefold: (1) It tiles the embedding matrix to increase data locality and reduce cache misses (thus reduce read latency); (2) It optimizes stochastic gradient descent (SGD) with sampling by parallelizing vector products instead of matrix-matrix multiplications, in particular the similarity computation therein, to avoid memory copies for matrix data preparation; and (3) It aggressively reuses intermediate results from the forward phase in the backward phase to alleviate redundant computation. Evaluation on five widely used datasets with both x86- and ARM-architecture processors shows that HEAT achieves up to 65.3X speedup over existing CPU solution and 4.8X speedup and 7.9X cost reduction in Cloud over existing GPU solution with NVIDIA V100 GPU.

翻译：协同过滤已被证明是最有效的推荐技术之一。在所有协同过滤方法中，SimpleX 是当前最先进的方法，采用了新型损失函数和适当数量的负样本。然而，目前尚无研究针对多核 CPU 优化 SimpleX，导致其性能受限。为此，我们对现有 SimpleX 实现进行了深度剖析与性能瓶颈分析，发现其主要问题包括：(1) 不规则内存访问、(2) 不必要的内存复制和(3) 冗余计算。针对这些问题，我们提出了一种高效协同过滤训练系统（称为 HEAT），充分挖掘现代 CPU 的多级缓存与多线程能力。具体而言，HEAT 的优化体现在三方面：(1) 通过分块嵌入矩阵提升数据局部性并减少缓存未命中（从而降低读取延迟）；(2) 通过并行化向量乘积（而非矩阵-矩阵乘法）优化带采样的随机梯度下降，特别是其中的相似度计算，以避免矩阵数据准备过程中的内存复制；(3) 在前向传播阶段激进地复用中间结果用于反向阶段，以减少冗余计算。在五种广泛使用的数据集上，基于 x86 和 ARM 架构处理器的评估表明，相比现有 CPU 方案，HEAT 实现高达 65.3 倍的加速；而在云环境下，相比采用 NVIDIA V100 GPU 的现有 GPU 方案，HEAT 实现 4.8 倍加速和 7.9 倍成本降低。