Historical user-item interaction datasets are essential in training modern recommender systems for predicting user preferences. However, the arbitrary user behaviors in most recommendation scenarios lead to a large volume of noisy data instances being recorded, which cannot fully represent their true interests. While a large number of denoising studies are emerging in the recommender system community, all of them suffer from highly dynamic data distributions. In this paper, we propose a Deep Reinforcement Learning (DRL) based framework, AutoDenoise, with an Instance Denoising Policy Network, for denoising data instances with an instance selection manner in deep recommender systems. To be specific, AutoDenoise serves as an agent in DRL to adaptively select noise-free and predictive data instances, which can then be utilized directly in training representative recommendation models. In addition, we design an alternate two-phase optimization strategy to train and validate the AutoDenoise properly. In the searching phase, we aim to train the policy network with the capacity of instance denoising; in the validation phase, we find out and evaluate the denoised subset of data instances selected by the trained policy network, so as to validate its denoising ability. We conduct extensive experiments to validate the effectiveness of AutoDenoise combined with multiple representative recommender system models.

翻译：历史用户-物品交互数据集对于训练现代推荐系统以预测用户偏好至关重要。然而，大多数推荐场景中用户行为的随意性导致大量噪声数据实例被记录，这些实例无法完全代表用户的真实兴趣。尽管推荐系统领域涌现出大量去噪研究，但它们都受限于高度动态的数据分布。本文提出一种基于深度强化学习的框架AutoDenoise，配备实例去噪策略网络，通过实例选择机制对深度推荐系统中的数据实例进行去噪。具体而言，AutoDenoise作为深度强化学习中的智能体，自适应地选择无噪声且具有预测能力的数据实例，这些实例可直接用于训练代表性推荐模型。此外，我们设计了一种交替两阶段优化策略来合理训练和验证AutoDenoise：在搜索阶段，旨在训练具备实例去噪能力的策略网络；在验证阶段，识别并评估由训练后的策略网络选择的去噪数据实例子集，以验证其去噪能力。我们进行了大量实验，验证了AutoDenoise结合多种代表性推荐系统模型的有效性。