Click-through rate (CTR) prediction is a core task in recommender systems. Existing methods (IDRec for short) rely on unique identities to represent distinct users and items that have prevailed for decades. On one hand, IDRec often faces significant performance degradation on cold-start problem; on the other hand, IDRec cannot use longer training data due to constraints imposed by iteration efficiency. Most prior studies alleviate the above problems by introducing pre-trained knowledge(e.g. pre-trained user model or multi-modal embeddings). However, the explosive growth of online latency can be attributed to the huge parameters in the pre-trained model. Therefore, most of them cannot employ the unified model of end-to-end training with IDRec in industrial recommender systems, thus limiting the potential of the pre-trained model. To this end, we propose a $\textbf{P}$re-trained $\textbf{P}$lug-in CTR $\textbf{M}$odel, namely PPM. PPM employs multi-modal features as input and utilizes large-scale data for pre-training. Then, PPM is plugged in IDRec model to enhance unified model's performance and iteration efficiency. Upon incorporating IDRec model, certain intermediate results within the network are cached, with only a subset of the parameters participating in training and serving. Hence, our approach can successfully deploy an end-to-end model without causing huge latency increases. Comprehensive offline experiments and online A/B testing at JD E-commerce demonstrate the efficiency and effectiveness of PPM.

翻译：摘要：点击率（CTR）预测是推荐系统中的核心任务。现有方法（简称IDRec）依赖唯一标识符来表征不同用户和物品，这一方法已盛行数十年。一方面，IDRec在冷启动问题上常出现显著的性能下降；另一方面，受迭代效率限制，IDRec无法利用更长的训练数据。以往多数研究通过引入预训练知识（如预训练用户模型或多模态嵌入）来缓解上述问题。然而，预训练模型中的海量参数导致在线延迟呈爆炸式增长。因此，大多数方法无法在工业推荐系统中与IDRec实现端到端联合训练的统一模型，从而限制了预训练模型的潜力。为此，我们提出了一种$\textbf{预}$训练$\textbf{插}$件CTR$\textbf{模}$型，即PPM。PPM采用多模态特征作为输入，并利用大规模数据进行预训练。随后，PPM被插入IDRec模型中以增强统一模型的性能与迭代效率。在集成IDRec模型后，网络内的部分中间结果被缓存，仅需部分参数参与训练与服务。因此，我们的方法能够成功部署端到端模型，且不会引起严重的延迟增加。在京东电商平台上进行的全面离线实验与在线A/B测试证明了PPM的高效性与有效性。