Relevance is generally understood as a multi-level and multi-dimensional relationship between an information need and an information object. However, traditional IR evaluation metrics naively assume mono-dimensionality. We ask: How to deal with multidimensional and graded relevance assessments in IR evaluation? Moreover, search result evaluation metrics neglect document overlaps and naively assume gains piling up as the searcher examines the ranked list into greater length. Consequently, we examine: How to deal with document overlap in IR evaluation? The usability of a document for a person-in-need also depends on document usability attributes beyond relevance. Therefore, we ask: How to deal with usability attributes, and how to combine this with multidimensional relevance assessments in IR evaluation? Finally, we ask how to define a formal model, which deals with multidimensional graded relevance assessments, document overlaps, and document usability attributes in a coherent framework serving IR evaluation?

翻译：相关性通常被理解为信息需求与信息对象之间的多层级、多维度关系。然而，传统的信息检索评估指标却简单地假设其具有单维度性。我们提出疑问：在信息检索评估中，应如何处理多维度分级相关性评估？此外，搜索结果评估指标忽略了文档重叠问题，并简单假设随着检索者深入浏览排序列表，收益会持续累积。因此，我们探讨：在信息检索评估中，应如何处理文档重叠？文档对需求者的可用性还取决于相关性之外的文档可用性属性。基于此，我们追问：在信息检索评估中，应如何处理可用性属性，以及如何将其与多维度相关性评估相结合？最终，我们探究如何构建一个形式化模型，该模型能够在一个连贯的框架内综合处理多维度分级相关性评估、文档重叠及文档可用性属性，从而服务于信息检索评估？