In the realm of deep learning-based recommendation systems, the increasing computational demands, driven by the growing number of users and items, pose a significant challenge to practical deployment. This challenge is primarily twofold: reducing the model size while effectively learning user and item representations for efficient recommendations. Despite considerable advancements in model compression and architecture search, prevalent approaches face notable constraints. These include substantial additional computational costs from pre-training/re-training in model compression and an extensive search space in architecture design. Additionally, managing complexity and adhering to memory constraints is problematic, especially in scenarios with strict time or space limitations. Addressing these issues, this paper introduces a novel learning paradigm, Dynamic Sparse Learning (DSL), tailored for recommendation models. DSL innovatively trains a lightweight sparse model from scratch, periodically evaluating and dynamically adjusting each weight's significance and the model's sparsity distribution during the training. This approach ensures a consistent and minimal parameter budget throughout the full learning lifecycle, paving the way for "end-to-end" efficiency from training to inference. Our extensive experimental results underline DSL's effectiveness, significantly reducing training and inference costs while delivering comparable recommendation performance.

翻译：在基于深度学习的推荐系统领域中，由用户和物品数量不断增长驱动的计算需求日益增加，对实际部署构成了重大挑战。这一挑战主要体现在两个方面：在有效学习用户和物品表示以实现高效推荐的同时，减小模型大小。尽管模型压缩与架构搜索方面取得了显著进展，但现有方法面临诸多显著限制。这些限制包括模型压缩中预训练/再训练带来的大量额外计算成本，以及架构设计中庞大的搜索空间。此外，在严格的时间或空间限制场景下，管理复杂性和满足内存约束也充满困难。针对这些问题，本文提出了一种专为推荐模型设计的新型学习范式——动态稀疏学习（DSL）。DSL创新性地从头开始训练轻量级稀疏模型，在训练过程中定期评估并动态调整每个权重的重要性以及模型的稀疏分布。该方法确保在整个学习生命周期中保持一致且最小化的参数预算，为从训练到推理的“端到端”效率铺平了道路。我们广泛的实验结果突显了DSL的有效性，在显著降低训练和推理成本的同时，提供了可比的推荐性能。