Explanations are crucial for improving users' transparency, persuasiveness, engagement, and trust in Recommender Systems (RSs). However, evaluating the effectiveness of explanation algorithms regarding those goals remains challenging due to existing offline metrics' limitations. This paper introduces new metrics for the evaluation and validation of explanation algorithms based on the items and properties used to form the sentence of an explanation. Towards validating the metrics, the results of three state-of-the-art post-hoc explanation algorithms were evaluated for six RSs, comparing the offline metrics results with those of an online user study. The findings show the proposed offline metrics can effectively measure the performance of explanation algorithms and highlight a trade-off between the goals of transparency and trust, which are related to popular properties, and the goals of engagement and persuasiveness, which are associated with the diversification of properties displayed to users. Furthermore, the study contributes to the development of more robust evaluation methods for explanation algorithms in RSs.

翻译：解释对于提升推荐系统中用户的透明度、说服力、参与度和信任度至关重要。然而，由于现有离线指标的局限性，评估解释算法在这些目标上的有效性仍具挑战。本文引入基于构成解释句子的项目及属性的新指标，用于评估和验证解释算法。为验证指标有效性，我们对六种推荐系统使用了三种最先进的局部事后解释算法，并将离线指标结果与在线用户研究结果进行对比。实验表明，所提出的离线指标能有效衡量解释算法性能，并揭示了透明度与信任目标（与常见属性相关）及参与度与说服力目标（与向用户展示的属性多样性相关）之间的权衡关系。此外，本研究为推荐系统中解释算法更鲁棒评估方法的开发做出了贡献。