The increasing popularity of deep learning models has created new opportunities for developing AI-based recommender systems. Designing recommender systems using deep neural networks requires careful architecture design, and further optimization demands extensive co-design efforts on jointly optimizing model architecture and hardware. Design automation, such as Automated Machine Learning (AutoML), is necessary to fully exploit the potential of recommender model design, including model choices and model-hardware co-design strategies. We introduce a novel paradigm that utilizes weight sharing to explore abundant solution spaces. Our paradigm creates a large supernet to search for optimal architectures and co-design strategies to address the challenges of data multi-modality and heterogeneity in the recommendation domain. From a model perspective, the supernet includes a variety of operators, dense connectivity, and dimension search options. From a co-design perspective, it encompasses versatile Processing-In-Memory (PIM) configurations to produce hardware-efficient models. Our solution space's scale, heterogeneity, and complexity pose several challenges, which we address by proposing various techniques for training and evaluating the supernet. Our crafted models show promising results on three Click-Through Rates (CTR) prediction benchmarks, outperforming both manually designed and AutoML-crafted models with state-of-the-art performance when focusing solely on architecture search. From a co-design perspective, we achieve 2x FLOPs efficiency, 1.8x energy efficiency, and 1.5x performance improvements in recommender models.

翻译：深度学习模型的日益普及为开发基于人工智能的推荐系统创造了新的机遇。使用深度神经网络设计推荐系统需要精心的架构设计，而进一步优化则需在联合优化模型架构与硬件方面进行大量协同设计工作。为充分挖掘推荐模型设计的潜力（包括模型选择及模型-硬件协同设计策略），设计自动化技术（如自动化机器学习）是必要的。我们提出一种利用权重共享探索丰富解空间的新范式。该范式创建了一个大型超网络，以搜索最优架构和协同设计策略，从而应对推荐领域中数据多模态与异构性带来的挑战。从模型视角看，该超网络包含多种算子、密集连接及维度搜索选项；从协同设计视角看，它涵盖多样化的存内处理配置以生成硬件高效的模型。我们解空间的规模、异构性与复杂性带来了若干挑战，我们通过提出多种超网络训练与评估技术应对这些挑战。在仅关注架构搜索时，我们构建的模型在三个点击率预测基准测试中展现出优异结果，其性能超越了人工设计模型与自动化机器学习构建的模型，达到了最先进水平。从协同设计视角看，我们在推荐模型中实现了2倍浮点运算效率提升、1.8倍能效提升及1.5倍性能改进。