The effectiveness of an IR system is gauged not just by its ability to retrieve relevant results but also by how it presents these results to users; an engaging presentation often correlates with increased user satisfaction. While existing research has delved into the link between user satisfaction, IR performance metrics, and presentation, these aspects have typically been investigated in isolation. Our research aims to bridge this gap by examining the relationship between query performance, presentation and user satisfaction. For our analysis, we conducted a between-subjects experiment comparing the effectiveness of various result card layouts for an ad-hoc news search interface. Drawing data from the TREC WaPo 2018 collection, we centered our study on four specific topics. Within each of these topics, we assessed six distinct queries with varying nDCG values. Our study involved 164 participants who were exposed to one of five distinct layouts containing result cards, such as "title'', "title+image'', or "title+image+summary''. Our findings indicate that while nDCG is a strong predictor of user satisfaction at the query level, there exists no linear relationship between the performance of the query, presentation of results and user satisfaction. However, when considering the total gain on the initial result page, we observed that presentation does play a significant role in user satisfaction (at the query level) for certain layouts with result cards such as, title+image or title+image+summary. Our results also suggest that the layout differences have complex and multifaceted impacts on satisfaction. We demonstrate the capacity to equalize user satisfaction levels between queries of varying performance by changing how results are presented. This emphasizes the necessity to harmonize both performance and presentation in IR systems, considering users' diverse preferences.

翻译：信息检索系统的有效性不仅取决于其检索相关结果的能力，还取决于向用户呈现这些结果的方式；富有吸引力的演示往往与更高的用户满意度相关。现有研究虽已探讨用户满意度、信息检索系统性能指标及演示之间的关联，但这些方面通常被孤立研究。本研究旨在填补这一空白，通过分析查询性能、演示与用户满意度之间的关系。我们采用被试间实验设计，针对临时新闻搜索界面比较了不同结果卡片布局的效果。实验数据源自TREC WaPo 2018数据集，重点围绕四个特定主题展开。在每个主题下，我们评估了六种不同归一化折损累计增益值的查询。研究共招募164名参与者，采用五种不同的结果卡片布局（如"标题"、"标题+图像"或"标题+图像+摘要"）。研究结果表明，尽管归一化折损累计增益在查询层面对用户满意度具有强预测能力，但查询性能、结果呈现与用户满意度之间不存在线性关系。然而，在考虑初始结果页面的总增益时，我们发现对于某些结果卡片布局（如"标题+图像"或"标题+图像+摘要"），演示方式在查询层面对用户满意度具有显著影响。结果亦表明，布局差异对满意度产生复杂且多维的影响。我们证明了通过改变结果呈现方式，可均衡不同性能查询之间的用户满意度水平。这强调了信息检索系统中必须协调性能与演示，并考虑用户多样化偏好的必要性。