Conversational recommender systems (CRSs) are improving rapidly, according to the standard recommendation accuracy metrics. However, it is essential to make sure that these systems are robust in interacting with users including regular and malicious users who want to attack the system by feeding the system modified input data. In this paper, we propose an adversarial evaluation scheme including four scenarios in two categories and automatically generate adversarial examples to evaluate the robustness of these systems in the face of different input data. By executing these adversarial examples we can compare the ability of different conversational recommender systems to satisfy the user's preferences. We evaluate three CRSs by the proposed adversarial examples on two datasets. Our results show that none of these systems are robust and reliable to the adversarial examples.

翻译：对话推荐系统（CRSs）正根据标准推荐准确性指标而迅速改进。然而，确保这些系统在与用户（包括试图通过输入修改数据来攻击系统的常规用户及恶意用户）交互时具有鲁棒性至关重要。本文提出了一种对抗性评估方案，涵盖两类共四种场景，并自动生成对抗样本来评估这些系统面对不同输入数据时的鲁棒性。通过执行这些对抗样本，我们可以比较不同对话推荐系统满足用户偏好的能力。我们利用所提出的对抗样本在两个数据集上对三种CRSs进行了评估。结果表明，这些系统均不具备对抗样本的鲁棒性与可靠性。