The Click-Through Rate (CTR) prediction task is critical in industrial recommender systems, where models are usually deployed on dynamic streaming data in practical applications. Such streaming data in real-world recommender systems face many challenges, such as distribution shift, temporal non-stationarity, and systematic biases, which bring difficulties to the training and utilizing of recommendation models. However, most existing studies approach the CTR prediction as a classification task on static datasets, assuming that the train and test sets are independent and identically distributed (a.k.a, i.i.d. assumption). To bridge this gap, we formulate the CTR prediction problem in streaming scenarios as a Streaming CTR Prediction task. Accordingly, we propose dedicated benchmark settings and metrics to evaluate and analyze the performance of the models in streaming data. To better understand the differences compared to traditional CTR prediction tasks, we delve into the factors that may affect the model performance, such as parameter scale, normalization, regularization, etc. The results reveal the existence of the ''streaming learning dilemma'', whereby the same factor may have different effects on model performance in the static and streaming scenarios. Based on the findings, we propose two simple but inspiring methods (i.e., tuning key parameters and exemplar replay) that significantly improve the effectiveness of the CTR models in the new streaming scenario. We hope our work will inspire further research on streaming CTR prediction and help improve the robustness and adaptability of recommender systems.

翻译：点击率（CTR）预测任务是工业推荐系统中的关键环节，在实际应用中，模型通常部署在动态的流式数据上。真实世界推荐系统中的这类流式数据面临着诸多挑战，例如分布漂移、时序非平稳性和系统性偏差，这些问题给推荐模型的训练与应用带来了困难。然而，现有大多数研究将CTR预测视为静态数据集上的分类任务，假设训练集与测试集独立同分布（即i.i.d.假设）。为弥合这一差距，我们将流式场景下的CTR预测问题形式化为流式CTR预测任务。据此，我们提出了专门的基准设置与评估指标，用于衡量和分析模型在流式数据上的表现。为更好地理解其与传统CTR预测任务的差异，我们深入探究了可能影响模型性能的因素，如参数规模、归一化、正则化等。研究结果揭示了"流式学习困境"的存在，即同一因素在静态与流式场景下可能对模型性能产生不同影响。基于这些发现，我们提出了两种简单但富有启发性的方法（即调整关键参数与示例重放），这些方法显著提升了CTR模型在新流式场景下的有效性。我们期望此项工作能激发对流式CTR预测的进一步研究，并助力提升推荐系统的鲁棒性与适应性。