Deep learning-based recommendation systems (e.g., DLRMs) are widely used AI models to provide high-quality personalized recommendations. Training data used for modern recommendation systems commonly includes categorical features taking on tens-of-millions of possible distinct values. These categorical tokens are typically assigned learned vector representations, that are stored in large embedding tables, on the order of 100s of GB. Storing and accessing these tables represent a substantial burden in commercial deployments. Our work proposes MEM-REC, a novel alternative representation approach for embedding tables. MEM-REC leverages bloom filters and hashing methods to encode categorical features using two cache-friendly embedding tables. The first table (token embedding) contains raw embeddings (i.e. learned vector representation), and the second table (weight embedding), which is much smaller, contains weights to scale these raw embeddings to provide better discriminative capability to each data point. We provide a detailed architecture, design and analysis of MEM-REC addressing trade-offs in accuracy and computation requirements, in comparison with state-of-the-art techniques. We show that MEM-REC can not only maintain the recommendation quality and significantly reduce the memory footprint for commercial scale recommendation models but can also improve the embedding latency. In particular, based on our results, MEM-REC compresses the MLPerf CriteoTB benchmark DLRM model size by 2900x and performs up to 3.4x faster embeddings while achieving the same AUC as that of the full uncompressed model.

翻译：摘要：基于深度学习的推荐系统（如DLRMs）被广泛用作提供高质量个性化推荐的人工智能模型。现代推荐系统的训练数据通常包含拥有数千万可能不同取值的类别特征。这些类别令牌通常被分配学习所得的向量表征，并以大规模嵌入表的形式存储，表规模可达数百GB。在商业部署中，存储和访问这些表造成了沉重负担。我们的工作提出了MEM-REC，一种新颖的嵌入表替代表征方法。MEM-REC利用布隆过滤器和哈希方法，通过两个缓存友好的嵌入表对类别特征进行编码：第一个表（令牌嵌入表）包含原始嵌入（即学习所得的向量表征），第二个表（权重嵌入表）则远小于前者，包含用于缩放这些原始嵌入的权重，从而为每个数据点提供更强的区分能力。我们提供了MEM-REC的详细架构、设计与分析，并与现有最优技术进行了精度与计算需求之间的权衡对比。研究表明，MEM-REC不仅能在商业规模推荐模型中保持推荐质量并显著减少内存占用，还能改善嵌入延迟。具体而言，根据我们的结果，MEM-REC将MLPerf CriteoTB基准DLRM模型大小压缩了2900倍，嵌入速度提升最高达3.4倍，同时实现了与未压缩完整模型相同的AUC。