Traditional click-through rate (CTR) prediction models convert the tabular data into one-hot vectors and leverage the collaborative relations among features for inferring user's preference over items. This modeling paradigm discards the essential semantic information. Though some recent works like P5 and M6-Rec have explored the potential of using Pre-trained Language Models (PLMs) to extract semantic signals for CTR prediction, they are computationally expensive and suffer from low efficiency. Besides, the beneficial collaborative relations are not considered, hindering the recommendation performance. To solve these problems, in this paper, we propose a novel framework \textbf{CTRL}, which is industrial friendly and model-agnostic with high training and inference efficiency. Specifically, the original tabular data is first converted into textual data. Both tabular data and converted textual data are regarded as two different modalities and are separately fed into the collaborative CTR model and pre-trained language model. A cross-modal knowledge alignment procedure is performed to fine-grained align and integrate the collaborative and semantic signals, and the lightweight collaborative model can be deployed online for efficient serving after fine-tuned with supervised signals. Experimental results on three public datasets show that CTRL outperforms the SOTA CTR models significantly. Moreover, we further verify its effectiveness on a large-scale industrial recommender system.

翻译：摘要：传统点击率（CTR）预测模型将表格数据转化为独热向量，并利用特征间的协同关系推断用户对物品的偏好。这种建模范式丢失了关键的语义信息。尽管近期P5和M6-Rec等工作尝试利用预训练语言模型（PLMs）提取CTR预测的语义信号，但它们计算开销大且效率低下。此外，有益的协同关系未被纳入考量，限制了推荐性能。为解决这些问题，本文提出新型框架\textbf{CTRL}，该框架具有工业友好性、模型无关性，且训练和推理效率高。具体而言，原始表格数据首先转化为文本数据。表格数据和转化后的文本数据被视为两种不同模态，分别输入协同CTR模型和预训练语言模型。通过跨模态知识对齐流程，对协同信号和语义信号进行细粒度对齐与整合，经监督信号微调后，轻量级协同模型可部署在线进行高效服务。三个公开数据集上的实验结果表明，CTRL显著优于当前最先进的CTR模型。此外，我们在大规模工业推荐系统中进一步验证了其有效性。