The Probability Ranking Principle (PRP) ranks search results based on their expected utility derived solely from document contents, often overlooking the nuances of presentation and user interaction. However, with the evolution of Search Engine Result Pages (SERPs), now comprising a variety of result cards, the manner in which these results are presented is pivotal in influencing user engagement and satisfaction. This shift prompts the question: How does the PRP and its user-centric counterpart, the Interactive Probability Ranking Principle (iPRP), compare in the context of these heterogeneous SERPs? Our study draws a comparison between the PRP and the iPRP, revealing significant differences in their output. The iPRP, accounting for item-specific costs and interaction probabilities to determine the ``Expected Perceived Utility" (EPU), yields different result orderings compared to the PRP. We evaluate the effect of the EPU on the ordering of results by observing changes in the ranking within a heterogeneous SERP compared to the traditional ``ten blue links''. We find that changing the presentation affects the ranking of items according to the (iPRP) by up to 48\% (with respect to DCG, TBG and RBO) in ad-hoc search tasks on the TREC WaPo Collection. This work suggests that the iPRP should be employed when ranking heterogeneous SERPs to provide a user-centric ranking that adapts the ordering based on the presentation and user engagement.
翻译:概率排序原则(PRP)仅基于文档内容计算预期效用进行结果排序,往往忽略了用户交互与呈现方式的细微差异。然而,随着搜索引擎结果页面(SERPs)演变为包含多种结果卡片的结构,这些结果的呈现方式对用户参与度和满意度的影响变得至关重要。这一转变引出了一个关键问题:在异构SERPs的背景下,PRP及其以用户为中心的变体——交互概率排序原则(iPRP)——具有怎样的差异?本研究对PRP与iPRP进行了比较分析,发现两者生成的排序存在显著差异。iPRP通过考虑项目特定成本和交互概率计算"预期感知效用"(EPU),其生成的结果排序与PRP存在本质区别。我们通过观察异构SERP与传统"十条蓝色链接"排序结果的变化,评估EPU对结果排序的影响。实验表明,在TREC WaPo数据集上的即席搜索任务中,根据iPRP原则变更呈现方式会导致项目排序发生高达48%的变化(基于DCG、TBG和RBO指标)。本研究表明,在排序异构SERPs时应采用iPRP原则,从而根据呈现方式和用户交互动态调整排序,实现以用户为中心的排序策略。