Conversational recommender systems (CRSs) are improving rapidly, according to the standard recommendation accuracy metrics. However, it is essential to make sure that these systems are robust in interacting with users including regular and malicious users who want to attack the system by feeding the system modified input data. In this paper, we propose an adversarial evaluation scheme including four scenarios in two categories and automatically generate adversarial examples to evaluate the robustness of these systems in the face of different input data. By executing these adversarial examples we can compare the ability of different conversational recommender systems to satisfy the user's preferences. We evaluate three CRSs by the proposed adversarial examples on two datasets. Our results show that none of these systems are robust and reliable to the adversarial examples.

翻译：根据标准建议准确度衡量标准,对话建议系统(CRSs)正在迅速改善,但必须确保这些系统在与用户,包括想要通过输入系统修改输入数据攻击系统的经常用户和恶意用户的互动中保持稳健。在本文件中,我们提出了一个对抗性评价计划,包括分为两类的四种设想方案,并自动生成对抗性实例,以在不同的输入数据面前评价这些系统的稳健性。通过执行这些对抗性实例,我们可以比较不同对话建议系统满足用户偏好的能力。我们用两个数据集的拟议对抗性实例对三个CRS进行了评估。我们的结果显示,这些系统中没有一个系统对对抗性实例是健全和可靠的。</s>