Ranking is at the core of Information Retrieval. Classic ranking optimization studies often treat ranking as a sorting problem with the assumption that the best performance of ranking would be achieved if we rank items according to their individual utility. Accordingly, considerable ranking metrics have been developed and learning-to-rank algorithms that have been designed to optimize these simple performance metrics have been widely used in modern IR systems. As applications evolve, however, people's need for information retrieval have shifted from simply retrieving relevant documents to more advanced information services that satisfy their complex working and entertainment needs. Thus, more complicated and user-centric objectives such as user satisfaction and engagement have been adopted to evaluate modern IR systems today. Those objectives, unfortunately, are difficult to be optimized under existing learning-to-rank frameworks as they are subject to great variance and complicated structures that cannot be explicitly explained or formulated with math equations like those simple performance metrics. This leads to the following research question -- how to optimize result ranking for complex ranking metrics without knowing their internal structures? To address this question, we conduct formal analysis on the limitation of existing ranking optimization techniques and describe three research tasks in \textit{Metric-agnostic Ranking Optimization}. Through the discussion of potential solutions to these tasks, we hope to encourage more people to look into the problem of ranking optimization in complex search and recommendation scenarios.

翻译：排序是信息检索的核心。经典的排序优化研究通常将排序视为一种排序问题，假设若根据项目的个体效用进行排序，便能实现最佳性能。据此，人们开发了大量排序指标，并设计了针对这些简单性能指标进行优化的学习排序算法，这些算法已广泛应用于现代信息检索系统。然而，随着应用的发展，用户对信息检索的需求已从单纯检索相关文档转向更高级的信息服务，以满足其复杂的工作和娱乐需求。因此，诸如用户满意度和参与度等更复杂且以用户为中心的目标现已成为评估现代信息检索系统的标准。遗憾的是，这些目标由于存在巨大方差且结构复杂，无法像简单性能指标那样用数学公式明确解释或表述，因此难以在现有学习排序框架下进行优化。这引出了以下研究问题——如何在不知其内部结构的情况下对复杂排序指标的结果排序进行优化？为解决此问题，我们形式化分析了现有排序优化技术的局限性，并在《与度量无关的排序优化》中描述了三个研究任务。通过探讨这些任务的可能解决方案，我们期望鼓励更多人关注复杂搜索和推荐场景中的排序优化问题。