The rise of deep neural networks offers new opportunities in optimizing recommender systems. However, optimizing recommender systems using deep neural networks requires delicate architecture fabrication. We propose NASRec, a paradigm that trains a single supernet and efficiently produces abundant models/sub-architectures by weight sharing. To overcome the data multi-modality and architecture heterogeneity challenges in the recommendation domain, NASRec establishes a large supernet (i.e., search space) to search the full architectures. The supernet incorporates versatile choice of operators and dense connectivity to minimize human efforts for finding priors. The scale and heterogeneity in NASRec impose several challenges, such as training inefficiency, operator-imbalance, and degraded rank correlation. We tackle these challenges by proposing single-operator any-connection sampling, operator-balancing interaction modules, and post-training fine-tuning. Our crafted models, NASRecNet, show promising results on three Click-Through Rates (CTR) prediction benchmarks, indicating that NASRec outperforms both manually designed models and existing NAS methods with state-of-the-art performance. Our work is publicly available at https://github.com/facebookresearch/NasRec.

翻译：深度神经网络的崛起为优化推荐系统带来了新机遇。然而，利用深度神经网络优化推荐系统需精心设计网络架构。我们提出NASRec范式，通过训练单个超网络并利用权重共享高效生成大量模型/子架构。为应对推荐领域中的数据多模态性与架构异构性挑战，NASRec构建了大型超网络（即搜索空间）以搜索完整架构。该超网络融合多样化的算子选择与密集连接设计，最大限度减少人工寻找先验知识的成本。NASRec的规模与异构性引发了训练低效、算子失衡及排序相关性下降等问题。我们通过提出单算子任意连接采样、算子均衡交互模块与训练后微调来解决这些挑战。基于NASRec生成的NASRecNet模型，在三个点击率（CTR）预测基准上展现出优异性能，表明NASRec在性能上超越人工设计模型及现有NAS方法。相关代码已开源至https://github.com/facebookresearch/NasRec。