Traditional click-through rate (CTR) prediction models convert the tabular data into one-hot vectors and leverage the collaborative relations among features for inferring the user's preference over items. This modeling paradigm discards essential semantic information. Though some works like P5 and CTR-BERT have explored the potential of using Pre-trained Language Models (PLMs) to extract semantic signals for CTR prediction, they are computationally expensive and suffer from low efficiency. Besides, the beneficial collaborative relations are not considered, hindering the recommendation performance. To solve these problems, in this paper, we propose a novel framework \textbf{CTRL}, which is industrial-friendly and model-agnostic with superior inference efficiency. Specifically, the original tabular data is first converted into textual data. Both tabular data and converted textual data are regarded as two different modalities and are separately fed into the collaborative CTR model and pre-trained language model. A cross-modal knowledge alignment procedure is performed to fine-grained align and integrate the collaborative and semantic signals, and the lightweight collaborative model can be deployed online for efficient serving after fine-tuned with supervised signals. Experimental results on three public datasets show that CTRL outperforms the state-of-the-art (SOTA) CTR models significantly. Moreover, we further verify its effectiveness on a large-scale industrial recommender system.

翻译：传统点击率预测模型将表格数据转换为独热编码向量，并利用特征间的协作关系推断用户对物品的偏好，但这种建模范式忽略了关键的语义信息。尽管P5、CTR-BERT等研究尝试利用预训练语言模型提取语义信号以提升点击率预测性能，但这些方法计算开销大且效率低下，同时未能充分考虑有益的协作关系，从而限制了推荐效果。为解决上述问题，本文提出一种工业友好且模型无关的新型框架CTRL，该框架兼具卓越的推理效率。具体而言，原始表格数据首先被转换为文本数据。表格数据与转换后的文本数据被视为两种不同模态，分别输入协作型点击率模型和预训练语言模型。通过跨模态知识对齐流程，可细粒度地整合协作信号与语义信号。经监督信号微调后，轻量级协作模型可部署于在线系统以实现高效服务。在三个公开数据集上的实验表明，CTRL显著优于当前最先进的点击率预测模型。此外，我们在大规模工业推荐系统中进一步验证了其有效性。